Display device and control method therefor

ABSTRACT

A method for controlling a display device, according to one embodiment of the present invention, comprises: displaying, on a main screen of a display device, content including at least one object; receiving an input for selecting a specific object from the at least one object displayed on the main screen; specifying the specific object by corresponding to the received selection input, expanding a specific area including the specific object; displaying a window including the content displayed on the main screen; displaying the expanded specific area on the main screen; and displaying, within the displayed window, an indicator for displaying the expended specific area, wherein the specific area changes according to a position of the specific object, and the content corresponds to a video.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/186,778, filed on Feb. 26, 2021, which is a continuation of U.S.patent application Ser. No. 15/573,806, filed on Nov. 13, 2017, now U.S.Pat. No. 10,986,302, which is the National Stage filing under 35 U.S.C.371 of International Application No. PCT/KR2015/00875, filed on Aug. 21,2015, which claims the benefit of earlier filing date and right ofpriority to Korean Application No. 10-2015-0068698, filed on May 18,2015, the contents of which are all hereby incorporated by referenceherein in their entirety.

TECHNICAL FIELD

The present invention relates to a display device and control methodtherefor, and more particularly, to a technique for specifying aspecific object such as a ball in an image on a smart TV or an STB,enlarging a specific area including the specified object, and displayingthe enlarged specific area.

BACKGROUND ART

Recently, a display device such as a smart TV has been used by manyusers. As the high-price customized content markets are extended andtypes thereof are diversified, the demands of the users for customizingimages also increase.

In the related art, a function of detecting an object that a userdesires to enlarge and watch has not been supported. Thus, when the userperforms the screen enlargement function on a video such as sports whereobject movement frequently occurs, the zoom screen should be movedwhenever a position of the object, which the user is interested in, ischanged.

For example, when a user desires to watch a soccer game focusing on asoccer ball and enlarge a screen at the center of the ball on a sportschannel broadcasting the soccer game, there may be a problem because thescreen cannot be partially enlarged in the related art. That is, sincewhen the ball moves, it is difficult to continuously trace the ballmovement, the user may experience inconvenience.

DISCLOSURE OF THE INVENTION Technical Task

In an embodiment of the present invention, provided are a display devicefor specifying a specific object such as a ball on a screen, enlarging aspecific area including the specified object, and displaying theenlarged specific area by continuously tracing the area and controlmethod therefor.

In another embodiment of the present invention, provided are a displaydevice for enabling a user to easily specify an object by displaying aboundary of the object as a guide line when a point approaches theobject and control method therefor.

In a further embodiment of the present invention, provided are a displaydevice for displaying a specific object on the center of a screen bypositioning the center of the specific object at the center of thescreen even when a user inputs no command through a remote controllerafter checking a position of the specific object and control methodtherefor.

In a still further embodiment of the present invention, provided are adisplay device for performing a zoom-out operation, i.e., going back tothe full screen when a displayed object moves out of a screen anddisplaying the object by zooming in on the object when the objectappears again on the screen and control method therefor.

Technical Solutions

In an aspect of the present invention, provided herein is a method forcontrolling a display device, including: displaying a content includingat least one object on a main screen of the display device; entering anenlargement mode in accordance with an input for requesting theenlargement mode received from a remote controller; receiving an inputfor selecting a specific object from among the at least one objectdisplayed on the main screen from the remote controller; specifying thespecific object in response to the received selection input; enlarging aspecific area including the specific object, wherein the specific areais changed according to a location of the specific object; displaying awindow including the content displayed on the main screen and displayingthe enlarged specific area on the main screen; and displaying anindicator indicating the enlarged specific area within the displayedwindow, wherein the content corresponds to a video.

In another aspect of the present invention, provided herein is a displaydevice, including: a display module configured to display a contentincluding at least one object on a main screen of the display device; aninterface module configured to receive an input for requestingenlargement and an input for selecting a specific object from among theat least one object displayed on the main screen from a remotecontroller; and a controller configured to control the display moduleand the interface module, wherein the controller is configured to enteran enlargement mode in accordance with the input for requesting theenlargement received from the remote controller, specify the specificobject in response to the received selection input, enlarge a specificarea including the specific object, wherein the specific area is changedaccording to a location of the specific object, display a windowincluding the content displayed on the main screen, display the enlargedspecific area on the main screen, and display an indicator indicatingthe enlarged specific area within the displayed window, and wherein thecontent corresponds to a video.

Advantageous Effects

According to an embodiment of the present invention, it is possible tospecify a specific object such as a ball on a screen, enlarge a specificarea including the specified object, and display the enlarged specificarea by continuously tracing the area, thereby improving userconvenience.

According to another embodiment of the present invention, when a pointapproaches an object, it is possible to enable a user to easily specifythe object by displaying a boundary of the object as a guide line,thereby improving user convenience.

According to a further embodiment of the present invention, even when auser inputs no command through a remote controller after checking aposition of a specific object, it is possible to display a specificobject on the center of a screen by positioning the center of thespecific object at the center of the screen, thereby improving userconvenience.

According to a still further embodiment of the present invention, when adisplayed object moves out of a screen, a zoom-out operation can beperformed, i.e., the full screen can be displayed. Thereafter, when theobject appears again on the screen, the object is displayed afterzooming in. That is, it is possible to provide a smooth screen switchingeffect, thereby improving user convenience.

DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram for explaining a service system includinga digital device according to an embodiment of the present invention.

FIG. 2 is a block diagram for explaining a digital device according toan embodiment of the present invention.

FIG. 3 is a block diagram for explaining a digital device according toanother embodiment of the present invention.

FIG. 4 is a block diagram for explaining a digital device according to afurther embodiment of the present invention.

FIG. 5 is a block diagram for explaining the detailed configuration ofeach of the controllers of FIGS. 2 to 4 according to an embodiment ofthe present invention.

FIG. 6 is a diagram illustrating an input unit connected to each of thedigital devices of FIGS. 2 to 4 according to an embodiment of thepresent invention.

FIG. 7 is a diagram for explaining a webOS architecture according to anembodiment of the present invention.

FIG. 8 is a diagram for explaining an architecture of a webOS deviceaccording to an embodiment of the present invention.

FIG. 9 is a diagram for explaining a graphic composition flow in a webOSdevice according to an embodiment of the present invention.

FIG. 10 is a diagram for explaining a media server according to anembodiment of the present invention.

FIG. 11 is a block diagram for explaining the configuration of a mediaserver according to an embodiment of the present invention.

FIG. 12 is a diagram for explaining a relationship between a mediaserver and a TV service according to an embodiment of the presentinvention.

FIG. 13 is a diagram illustrating a control method for a remote controldevice for controlling an arbitrary one among image display devicesaccording to embodiments of the present invention.

FIG. 14 is a block diagram illustrating the inside of a remote controldevice for controlling an arbitrary one among image display devicesaccording to embodiments of the present invention.

FIG. 15 is a block diagram illustrating the configuration of amultimedia device according to an embodiment of the present invention.

FIG. 16 is a flowchart for explaining a control method for a multimediadevice according to an embodiment of the present invention.

FIG. 17 is a diagram illustrating a case in which a mode for enlarging aspecific area (hereinafter referred to as a specific area enlargementmode) is activated according to an embodiment of the present invention.

FIG. 18 is a diagram illustrating a case in which a pointer shape ischanged when a specific area enlargement mode is activated according toan embodiment of the present invention.

FIG. 19 is a diagram illustrating an example of controlling a screenwhen a specific area enlargement mode is activated according to anembodiment of the present invention.

FIG. 20 is a diagram illustrating an example of moving a specific pointon an enlarged screen using a pointer when a specific area enlargementmode is activated according to an embodiment of the present invention.

FIG. 21 is a diagram illustrating an example of controlling a screenusing a remote controller when a specific area enlargement mode isactivated according to an embodiment of the present invention.

FIG. 22 is a diagram illustrating an example of automatically executinga specific area enlargement mode through association with EPGinformation according to an embodiment of the present invention.

FIG. 23 is a diagram illustrating an example of executing a specificarea enlargement mode through association with a time shift functionaccording to an embodiment of the present invention.

FIG. 24 is a diagram illustrating an example of switching between a fullscreen and a zoom screen according to an embodiment of the presentinvention.

FIG. 25 is a diagram illustrating an example of selecting a plurality ofpoints on a screen and enlarging the selected points according to anembodiment of the present invention.

FIG. 26 is a diagram illustrating an example of selecting a plurality ofpoints on a screen and enlarging the selected points according to anembodiment of the present invention.

FIG. 27 is a diagram illustrating a solution for a case in whichcoordinates from a remote controller do not match those of an inputvideo according to an embodiment of the present invention.

FIG. 28 is a diagram illustrating a solution for a case in which aspecific area to be enlarged is out of a video output range according toan embodiment of the present invention.

FIG. 29 is a diagram illustrating an example of dividing a screen into aprescribed number of partial screens, enlarging a selected screen if auser selects the screen from the divided screens, and displaying theenlarged screen when video data is outputted according to an embodimentof the present invention.

FIG. 30 is a diagram illustrating an example in which a controllerdivides a screen into four, nine, or sixteen partial screens accordingto selection from a user and controls the divided screens when videodata is outputted according to an embodiment of the present invention.

FIG. 31 is a diagram illustrating a process for adjusting amagnification ratio while a specific area enlargement mode is executedaccording to an embodiment of the present invention.

FIG. 32 is a diagram illustrating a process for selecting an area to beenlarged while a specific area enlargement mode is executed according toan embodiment of the present invention.

FIG. 33 is a diagram illustrating a process for eliminating a relatedindicator while a specific area enlargement mode is executed accordingto an embodiment of the present invention.

FIG. 34 is a diagram illustrating a process for displaying an eliminatedrelated indicator again while a specific area enlargement mode isexecuted according to an embodiment of the present invention.

FIG. 35 is a block diagram illustrating the configuration of a displaydevice according to an embodiment of the present invention.

FIG. 36 is a flowchart for explaining a control method for a displaydevice according to an embodiment of the present invention.

FIG. 37 is a diagram illustrating an example of specifying a specificobject and tracing the specified object according to an embodiment ofthe present invention.

FIG. 38 is a diagram illustrating an example of representing a boundaryof an object adjacent to a pointer as a dotted line when the pointermoves toward the object according to an embodiment of the presentinvention.

FIG. 39 is a diagram illustrating an example of representing a boundaryof an object adjacent to a pointer as a dotted line when the pointermoves toward the object according to an embodiment of the presentinvention.

FIG. 40 is a diagram illustrating a case in which the center of anobject is located at the center of a main screen according to anembodiment of the present invention.

FIG. 41 is a diagram illustrating a case in which an object moves out ofa screen according to an embodiment of the present invention.

FIG. 42 is a diagram illustrating a case in which an object that movesout of a screen appears again according to an embodiment of the presentinvention.

FIG. 43 is a diagram illustrating an example of specifying a differentobject and tracing the specified different object while displaying acontent according to an embodiment of the present invention.

FIG. 44 is a diagram illustrating a case in which an object has a smallsize or an object cannot be recognized according to an embodiment of thepresent invention.

FIG. 45 is a diagram illustrating an example of adjusting a zoom ratioand a position for each object according to an embodiment of the presentinvention.

FIG. 46 is a diagram illustrating an example of specifying multipleobjects and tracing the specified multiple objects according to anembodiment of the present invention.

FIG. 47 is a diagram illustrating an example of specifying multipleobjects and tracing the specified multiple objects according to anembodiment of the present invention.

FIG. 48 is a diagram illustrating an example of specifying any one amongmultiple objects and performing tracing with respect to the specifiedobject according to an embodiment of the present invention.

FIG. 49 is a diagram illustrating an example in which when an object isa person, the object is divided according to parts of the body and adifferent magnification ratio is applied depending on the selected partaccording to an embodiment of the present invention.

FIG. 50 is a diagram illustrating an example of specifying an object ina different manner depending on movement of the object according to anembodiment of the present invention.

FIG. 51 is a diagram illustrating an example in which when multipleobjects are specified, the specified multiple objects are displayed onPBP screens according to an embodiment of the present invention.

FIG. 52 is a diagram illustrating an example in which a user searchesfor a specific object on a content through association with time shiftand search functions according to an embodiment of the presentinvention.

BEST MODE FOR INVENTION

Description will now be given in detail according to variousembodiment(s) for a digital device and screen saver processing methodtherein disclosed herein, with reference to the accompanying drawings.

Suffixes such as “module”, “unit” and the like in this disclosure may beused to refer to elements or components. Use of such a suffix herein ismerely intended to facilitate description of the specification, and bothsuffixes may be interchangeably usable. The description with ordinalnumbers such as ‘first˜’, ‘second˜’ and the like is provided tofacilitate the description of the corresponding terminologies only,which is non-limited by such terminologies or ordinal numbers.

Although terminologies used in the present specification are selectedfrom general terminologies used currently and widely in consideration offunctions in the present invention, they may be changed in accordancewith intentions of technicians engaged in the corresponding fields,customs, advents of new technologies and the like. Occasionally, someterminologies may be arbitrarily selected by the applicant(s).

Therefore, the digital device, for example, on the universal OS kernel,it is available to add and delete various applications and variousfunctions can be performed.

In this case, the meanings of the arbitrarily selected terminologiesshall be described in the corresponding part of the detailed descriptionof the invention. Therefore, terminologies used in the presentspecification need to be construed based on the substantial meanings ofthe corresponding terminologies and the overall matters disclosed in thepresent specification rather than construed as simple names of theterminologies.

Meanwhile, the descriptions disclosed in the present specificationand/or drawings correspond to one preferred embodiment of the presentinvention and are non-limited by the preferred embodiment. And, thescope/extent of the right should be determined through the appendedclaims.

‘Digital device’ described in the present specification includes anydevice capable of performing at least one of transmission, reception,processing and output of data, content, service, application and thelike for example. The digital device can be paired or connected(hereinafter ‘paired’) with another digital device, an external serverand the like through wire/wireless network and transmit/receiveprescribed data through the pairing. In doing so, if necessary, the datamay be appropriately converted before the transmission/reception. Thedigital devices may include standing devices (e.g., Network TV, HBBTV(Hybrid Broadcast Broadband TV), Smart TV, IPTV (Internet Protocol TV),PC (Personal Computer), etc.) and mobile devices (e.g., PDA (PersonalDigital Assistant), Smart Phone, Tablet PC, Notebook, etc.). In thepresent specification, to help the understanding of the presentinvention and the clarity of the applicant's description, a digital TVand a mobile are shown as embodiments of digital devices in FIG. 2 andFIG. 3, respectively. A digital device described in the presentspecification may include a panel-only configuration, a configurationsuch as a set-top box (STB), or a single set configuration of device,system and the like.

Meanwhile, ‘wire/wireless network’ described in the presentspecification is a common name of a communication network supportive ofvarious communication specifications and/or protocols for the paringor/and data transceiving between digital devices or between a digitaldevice and an external server. Such wire/wireless networks include allcommunication networks supported currently or all communication networksthat will be supported in the future, by the specifications and arecapable of supporting one or more communication protocols for the same.Such wire/wireless networks can be established by a network for a wireconnection and a communication specification or protocol for the same(e.g., USB (Universal Serial Bus), CVBS (Composite Video Banking Sync),Component, S-video (analog), DVI (Digital Visual Interface), HDMI (HighDefinition Multimedia Interface), RGB, D-SUB, etc.) and a network for awireless connection and a communication specification or protocol (e.g.,Bluetooth, RFID (Radio Frequency Identification), IrDA (infrared DataAssociation), UWB (Ultra Wideband), ZigBee, DLNA (Digital Living NetworkAlliance), WLAN (Wireless LAN)(Wi-Fi), Wibro (Wireless broadband), Wimax(World Interoperability for Microwave Access), HSDPA (High SpeedDownlink Packet Access), LTE/LTE-A (Long Term Evolution/LTE-Advanced),Wi-Fi direct).

Besides, if a device is named a digital device in this disclosure, themeaning may indicate a standing device or a mobile device according to acontext, or can be used to indicate both unless mentioned specially.

Meanwhile, a digital device is an intelligent device supportive of abroadcast receiving function, a computer function or support, at leastone external input and the like, and is able to support e-mail, webbrowsing, banking, game, application and the like through theaforementioned wire/wireless network. Moreover, the digital device mayinclude an interface (e.g., manual input device, touchscreen, spaceremote controller, etc.) to support at least one input or control means.

Besides, a digital device may use a standardized OS (operating system).Particularly, a digital device described in the present specificationuses Web OS for one embodiment. Hence, a digital device can processadding, deleting, amending, updating and the like of various services orapplications on Universal OS kernel or Linux kernel, through which afurther user-friendly environment can be configured and provided.

Meanwhile, the aforementioned digital device can receive and process anexternal input. Herein, the external input includes an external inputdevice, i.e., any input means or digital device capable oftransmitting/receiving and processing data by being connected to theaforementioned digital device through wire/wireless network. Forinstance, as the external inputs, a game device (e.g., HDMI(High-Definition Multimedia Interface), Playstation, X-Box, etc.), aprinting device (e.g., smart phone, tablet PC, pocket photo, etc.), anda digital device (e.g., smart TV, Blu-ray device, etc.) are included.

Besides, ‘server’ described in the present specification means a digitaldevice or system that supplies data to the aforementioned digital device(i.e., client) or receives data from it, and may be called a processor.For example, the server may include a portal server providing web page,web content or web service, an advertising server providing advertisingdata, a content server providing contents, an SNS server providing SNS(Social Network Service), a service server provided by a manufacturer,an MVPD (Multichannel Video Programming Distributor) providing VoD(Video on Demand) or streaming service, a service server providing a payservice and the like.

Moreover, in case that the following description is made using anapplication only for clarity in the present specification, it may mean aservice as well as an application on the basis of a correspondingcontent and the like.

In the following description, the present invention is explained indetail with reference to attached drawings.

FIG. 1 is a schematic diagram illustrating a service system including adigital device according to one embodiment of the present invention.

Referring to FIG. 1, a service system may include a content provider(CP) 10, a service provider (SP) 20, a network provider (NP) 30, and ahome network end user (HNED) (Customer) 40. The HNED 40 includes aclient 100, that is, a digital device according to the presentinvention.

The CP 10 produces and provides various contents. Referring to FIG. 1,the CP 10 can include a terrestrial broadcaster, a cable system operator(SO), a multiple system operator (MSO), a satellite broadcaster, variousInternet broadcasters, private content providers (CPs), etc. Meanwhile,the CP 10 can produce and provide various services, applications and thelike as well as well as broadcast contents.

The SP 20 service-packetizes a content produced by the CP 10 and thenprovides it to the HNED 40. For instance, the SP 20 packetizes at leastone of contents, which are produced by a first terrestrial broadcaster,a second terrestrial broadcaster, a cable MSO, a satellite broadcaster,various internet broadcasters, applications and the like, for a serviceand then provides it to the HNED 40.

The SP 20 can provide services to the client 100 in a uni-cast ormulti-cast manner. Meanwhile, the SP 20 can collectively send data to amultitude of pre-registered clients 100. To this end, it is able to useIGMP (internet group management protocol) and the like.

The CP 10 and the SP 20 can be configured in the form of one entity. Forexample, the CP 10 can function as the SP 20 by producing a content,service-packetizing the produced content, and then providing it to theHNED 40, and vice versa.

The NP 30 provides a network environment for data exchange between theCP 10 and/or the SP 20 and the client 100.

The client 100 is a consumer belonging to the HNED 40. The client 100may receive data by establishing a home network through the NP 30 forexample and transmit/receive data for various services (e.g., VoD,streaming, etc.), applications and the like.

The CP 10 or/and the SP 20 in the service system may use a conditionalaccess or content protection means for the protection of a transmittedcontent. Hence, the client 100 can use a processing means such as acable card (CableCARD) (or POD (point of deployment) or a downloadableCAS (DCAS), which corresponds to the conditional access or the contentprotection.

In addition, the client 100 may use an interactive service through anetwork as well. In this case, the client 100 can directly serve as acontent provider. And, the SP 20 may receive and transmit it to anotherclient or the like.

In FIG. 1, the CP 10 or/and the SP 20 may be a service providing serverthat will be described later in the present specification. In this case,the server may mean that the NP 30 is owned or included if necessary. Inthe following description, despite not being specially mentioned, aservice or a service data includes an internal service or application aswell as a service or application received externally, and such a serviceor application may mean a service or application data for the Web OSbased client 100.

FIG. 2 is a block diagram showing a digital device according to oneembodiment of the present invention.

In the following, a digital device mentioned in the presentspecification may correspond to the client 100 shown in FIG. 1.

The digital device 200 may include a network interface 201, a TCP/IPmanager 202, a service delivery manager 203, an SI decoder 204, a demuxor demultiplexer 205, an audio decoder 206, a video decoder 207, adisplay A/V and OSD (On Screen Display) module 208, a service controlmanager 209, a service discovery manager 210, a SI & metadata database(DB) 211, a metadata manager 212, a service manager 213, a UI manager214, etc.

The network interface 201 may transmit/receive IP (internet protocol)packet(s) or IP datagram(s) (hereinafter named IP pack(s)) through anaccessed network. For instance, the network interface 201 may receiveservices, applications, contents and the like from the service provider20 shown in FIG. 1 through a network.

The TCP/IP manager 202 may involve delivery of IP packets transmitted tothe digital device 200 and IP packets transmitted from the digitaldevice 200, that is, packet delivery between a source and a destination.The TCP/IP manager 202 may classify received packet(s) according to anappropriate protocol and output the classified packet(s) to the servicedelivery manager 205, the service discovery manager 210, the servicecontrol manager 209, the metadata manager 212, and the like.

The service delivery manager 203 may be in charge of controlling thereceived service data. The service delivery manager 203 may controlreal-time streaming data, for example, using RTP/RTCP. In case oftransmitting the real-time streaming data using RTP, the servicedelivery manager 203 may parse the received data packet according to theRTP and then transmits the parsed data packet to the demultiplexer 205or save the parsed data packet to the SI & metadata DB 211 under thecontrol of the service manager 213. The service delivery manager 203 mayfeed back the network reception information to the service providingserver side using RTCP.

The demultiplexer 205 may demultiplex a received packet into audio data,video data, SI (system information) data and the like and then transmitthe demultiplexed data to the audio/video decoder 206/207 and the SIdecoder 204, respectively.

The SI decoder 204 may decode the demultiplexed SI data, i.e., serviceinformations of PSI (Program Specific Information), PSIP (Program andSystem Information Protocol), DVB-SI (Digital Video Broadcasting-ServiceInformation), DTMB/CMMB (Digital Television Terrestrial MultimediaBroadcasting/Coding Mobile Multimedia Broadcasting), etc. And, the SIdecoder 204 may save the decoded service informations to the SI &metadata DB 211. The saved service information can be used by being readby a corresponding component in response to a user's request forexample.

The audio decoder 206 and the video decoder 207 may decode thedemultiplexed audio data and the demultiplexed video data, respectively.The decoded audio and video data may be provided to the user through thedisplay unit 208.

The application manager includes a service manager 213 and a userinterface (UI) manager 214 and is able to perform a function of acontroller of the digital device 200. So to speak, the applicationmanager can administrate the overall states of the digital device 200,provide a user interface (UI), and manage other mangers.

The UI manager 214 provides a graphical user interface/user interface(GUI/UI) using OSD (on screen display) and the like. The UI manager 214receives a key input from a user and then performs a device operationaccording to the input. For instance, if receiving a key input about achannel selection from a user, the UI manager 214 transmits the keyinput signal to the service manager 213.

The service manager 213 may control and manage service-related managerssuch as the service delivery manager 203, the service discovery manager210, the service control manager 209, and the metadata manager 212.

The service manager 213 creates a channel map and controls a selectionof a channel and the like using the created channel map in response to akey input received from the UI manager 214. The service manager 213 mayreceive service information from the SI decoder 204 and then sets anaudio/video PID of a selected channel for the demultiplexer 205. Such aPID can be used for the demultiplexing procedure. Therefore, thedemultiplexer 205 performs filtering (PID or section filtering) on audiodata, video data and SI data using the PID.

The service discovery manager 210 may provide information required toselect a service provider that provides a service. Upon receipt of asignal for selecting a channel from the service manager 213, the servicediscovery manager 210 searches for a service using the information.

The service control manager 209 may select and control a service. Forexample, the service control manager 209 may perform service selectionand control using IGMP (Internet Group Management Protocol) or real timestreaming protocol (RTSP) when the user selects a live broadcast serviceand using RTSP when the user selects a video on demand (VOD) service.The RTSP protocol can provide a trick mode for real-time streaming. And,the service control manager 209 may initialize and manage a sessionthrough the IMS gateway 250 using IMS (IP multimedia subsystem) and SIP(session initiation protocol). The protocols are exemplary, and otherprotocols are usable according to implementations.

The metadata manager 212 may manage metadata associated with servicesand save the metadata to the SI & metadata DB 211.

The SI & metadata DB 211 may store service information decoded by the SIdecoder 204, metadata managed by the metadata manager 212, andinformation required to select a service provider, which is provided bythe service discovery manager 210. In addition, the SI & metadata DB 211can store system set-up data and the like for the system.

The SI & metadata database 211 may be implemented with non-volatile RAM(NVRAM), flash memory and the like.

Meanwhile, an IMS gateway 250 is a gateway in which functions requiredfor an access to an IMS based IPTV service are collected.

FIG. 3 is a block diagram to describe a digital device according toanother embodiment of the present invention.

The former description with reference to FIG. 2 is made by taking astanding device as one embodiment of a digital device. And, FIG. 3 usesa mobile device as another embodiment of a digital device.

Referring to FIG. 3, the mobile device 300 includes a wirelesscommunication unit 310, an A/V (audio/video) input unit 320, a userinput unit 330, a sensing unit 340, an output unit 350, a memory 360, aninterface unit 370, a controller 380, a power supply unit 390, etc.

The respective components are described in detail as follows.

The wireless communication unit 310 typically includes one or moremodules which permit wireless communication between the mobile device300 and a wireless communication system or network within which themobile device 300 is located. For instance, the wireless communicationunit 310 can include a broadcast receiving module 311, a mobilecommunication module 312, a wireless Internet module 313, a short-rangecommunication module 314, a location information module 315, etc.

The broadcast receiving module 311 receives a broadcast signal and/orbroadcast associated information from an external broadcast managingserver via a broadcast channel. The broadcast channel may include asatellite channel and a terrestrial channel. The broadcast managingserver may mean a server generating to send a broadcast signal and/orbroadcast associated information or a server receiving to send apre-generated broadcast signal and/or broadcast associated informationto a terminal. The broadcast signal may be implemented as a TV broadcastsignal, a radio broadcast signal, and/or a data broadcast signal, amongother signals. If desired, the broadcast signal may further include abroadcast signal combined with a TV or radio broadcast signal.

The broadcast associated information includes information associatedwith a broadcast channel, a broadcast program, or a broadcast serviceprovider. Furthermore, the broadcast associated information can beprovided via a mobile communication network. In this case, the broadcastassociated information can be received by the mobile communicationmodule 312.

The broadcast associated information can be implemented in variousforms, e.g., an electronic program guide (EPG), an electronic serviceguide (ESG), and the like.

The broadcast receiving module 311 may be configured to receive digitalbroadcast signals using broadcasting systems such as ATSC, DVB-T(Digital Video Broadcasting-Terrestrial), DVB-S (Satellite), MediaFLO(Media Forward Link Only), DVB-H (Handheld), ISDB-T (Integrated ServicesDigital Broadcast-Terrestrial), and the like. Optionally, the broadcastreceiving module 311 can be configured to be suitable for otherbroadcasting systems as well as the above-noted digital broadcastingsystems.

The broadcast signal and/or broadcast associated information received bythe broadcast receiving module 311 may be saved to the memory 360.

The mobile communication module 312 transmits/receives wireless signalsto/from at least one of a base station, an external terminal, and aserver via a mobile network. Such wireless signals may carry audiosignals, video signals, and data of various types according totransceived text/multimedia messages.

The wireless Internet module 313 includes a module for wireless Internetaccess and may be internally or externally coupled to the mobile device300. The wireless Internet technology can include WLAN (Wireless LAN)(Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability forMicrowave Access), HSDPA (High Speed Downlink Packet Access), and thelike.

The short-range communication module 314 is a module for short-rangecommunications. Suitable technologies for implementing this moduleinclude Bluetooth, radio frequency identification (RFID), infrared dataassociation (IrDA), ultra-wideband (UWB), ZigBee, RS-232, RS-485 and thelike.

The location information module 315 is a module for obtaining locationinformation of the mobile terminal 100. And, this module may beimplemented with a global positioning system (GPS) module for example.

The audio/video (A/V) input unit 320 is configured to provide audio orvideo signal input. The A/V input unit 320 may include a camera 321, amicrophone 322 and the like. The camera 321 receives and processes imageframes of still pictures or video, which are obtained by an image sensorin a video call mode or a photographing mode. Furthermore, the processedimage frames can be displayed on the display 351.

The image frames processed by the camera 321 can be stored in the memory360 or transmitted externally via the wireless communication unit 310.Optionally, at least two cameras 321 can be provided according to theenvironment of usage.

The microphone 322 receives an external audio signal in call mode,recording mode, voice recognition mode, or the like. This audio signalis processed and converted into electrical audio data. The processedaudio data is transformed into a format transmittable to a mobilecommunication base station via the mobile communication module 312 incall mode. The microphone 322 typically includes assorted noisecanceling algorithms to cancel noise generated in the course ofreceiving the external audio signal.

The user input unit 330 generates input data for a user to control anoperation of the terminal. The user input unit 330 may include a keypad,a dome switch, a touchpad (e.g., static pressure/capacitance), a jogwheel, a jog switch, and/or the like.

The sensing unit 340 generates sensing signals for controllingoperations of the mobile device 300 using status measurements of variousaspects of the mobile terminal. For instance, the sensing unit 340 maydetect an open/closed status of the mobile device 300, a location of themobile device 300, an orientation of the mobile device 300, a presenceor absence of user contact with the mobile device 300, anacceleration/deceleration of the mobile device 300, and the like. Forexample, if the mobile device 300 is moved or inclined, it is able tosense a location or inclination of the mobile device. Moreover, thesensing unit 340 may sense a presence or absence of power provided bythe power supply unit 390, a presence or absence of a coupling or otherconnection between the interface unit 370 and an external device, andthe like. Meanwhile, the sensing unit 340 may include a proximity sensor341 such as NFC (near field communication) and the like.

The output unit 350 generates output relevant to the senses of vision,hearing and touch, and may include the display 351, an audio outputmodule 352, an alarm unit 353, a haptic module 354, and the like.

The display 351 is typically implemented to visually display (output)information processed by the mobile device 300. For instance, if themobile terminal is operating in phone call mode, the display willgenerally provide a user interface (UI) or graphical user interface(GUI) related to a phone call. For another instance, if the mobiledevice 300 is in video call mode or photographing mode, the display 351may display photographed or/and received images or UI/GUI.

The display module 351 may be implemented using known displaytechnologies. These technologies include, for example, a liquid crystaldisplay (LCD), a thin film transistor-liquid crystal display (TFT-LCD),an organic light-emitting diode display (OLED), a flexible display and athree-dimensional display. The mobile device 300 may include one or moreof such displays.

Some of the displays can be implemented in a transparent or opticaltransmittive type, which can be called a transparent display. Arepresentative example of the transparent display is the TOLED(transparent OLED). A rear configuration of the display 351 can beimplemented as the optical transmittive type as well. In thisconfiguration, a user may be able to see an object located in rear of aterminal body through a region occupied by the display 351 of theterminal body.

Two or more displays 351 can be provided to the mobile device 300 inaccordance with an implementation type of the mobile device 300. Forinstance, a plurality of displays can be disposed on the mobile device300 in a manner of being spaced apart from a single face or beingintegrally formed on a single face. Alternatively, a plurality ofdisplays may be disposed on different faces of the mobile device 300,respectively.

If the display 351 and a sensor (hereinafter called ‘touch sensor’) fordetecting a touch action configure a mutual layer structure, the display351 is usable as an input device as well as an output device. In thiscase, the touch sensor can be configured with a touch film, a touchsheet, a touchpad, or the like.

The touch sensor can be configured to convert a pressure applied to aspecific portion of the display 351 or a variation of capacitancegenerated from a specific portion of the display 351 into an electricalinput signal. Moreover, the touch sensor is configurable to detectpressure of a touch as well as a touched position or size.

If a touch input is applied to the touch sensor, signal(s) correspondingto the touch input is transferred to a touch controller. The touchcontroller processes the signal(s) and then transfers the processedsignal(s) to the controller 380. Therefore, the controller 380 is ableto know whether a prescribed portion of the display 351 is touched.

A proximity sensor 341 can be disposed on an inner region of the mobiledevice enclosed by the touchscreen or near the touchscreen. Theproximity sensor is a sensor that detects a presence or non-presence ofan object approaching a prescribed detecting surface or an objectexisting around the proximity sensor using an electromagnetic fieldstrength or infrared ray without mechanical contact. Hence, theproximity sensor is more durable than a contact type sensor and also hasutility higher than that of the contact type sensor.

The proximity sensor may include one of a transmittive photoelectricsensor, a direct reflective photoelectric sensor, a mirror reflectivephotoelectric sensor, a radio frequency oscillation proximity sensor, anelectrostatic capacity proximity sensor, a magnetic proximity sensor, aninfrared proximity sensor, etc. If the touch screen includes theelectrostatic capacity proximity sensor, it is configured to detect theproximity of a pointer using a variation of an electric field accordingto the proximity of the pointer. In this configuration, the touchscreen(or touch sensor) can be sorted into a proximity sensor.

For clarity and convenience of explanation, an action for enabling thepointer approaching the touch screen to be recognized as placed on thetouch screen may be named ‘proximity touch’ and an action of enablingthe pointer to actually come into contact with the touch screen may benamed ‘contact touch’. And, a position, at which the proximity touch ismade to the touch screen using the pointer, may mean a position of thepointer vertically corresponding to the touch screen when the pointermakes the proximity touch.

The proximity sensor detects a proximity touch and a proximity touchpattern (e.g., a proximity touch distance, a proximity touch duration, aproximity touch position, a proximity touch shift state). Informationcorresponding to the detected proximity touch action and the detectedproximity touch pattern can be output to the touch screen.

The audio output module 352 functions in various modes including acall-receiving mode, a call-placing mode, a recording mode, a voicerecognition mode, and a broadcast reception mode to output audio datawhich is received from the wireless communication unit 310 or stored inthe memory 360. During operation, the audio output module 352 may outputan audio signal related to a function (e.g., call received, messagereceived) executed in the mobile device 300. The audio output module 352may include a receiver, a speaker, a buzzer and the like.

The alarm unit 353 outputs a signal for announcing the occurrence of anevent of the mobile device 300. Typical events occurring in the mobiledevice may include a call signal received, a message received, a touchinput received, and the like. The alarm unit 353 may output a signal forannouncing the event occurrence by way of vibration as well as video oraudio signal. The video or audio signal can be outputted via the display351 or the audio output module 352. Hence, the display 351 or the audiooutput module 352 can be sorted into a part of the alarm unit 353.

The haptic module 354 generates various tactile effects that can besensed by a user. Vibration is a representative one of the tactileeffects generated by the haptic module 354. The strength and pattern ofthe vibration generated by the haptic module 354 are controllable. Forinstance, different vibrations can be output in a manner of beingsynthesized together or can be output in sequence. The haptic module 354is able to generate various tactile effects as well as the vibration.For instance, the haptic module 354 may generate an effect attributed tothe arrangement of pins vertically moving against a contact skinsurface, an effect attributed to the injection/suction power of airthough an injection/suction hole, an effect attributed to the skim overa skin surface, an effect attributed to a contact with an electrode, aneffect attributed to an electrostatic force, and an effect attributed tothe representation of a hot/cold sense using an endothermic orexothermic device. The haptic module 354 can be implemented to enable auser to sense the tactile effect through a muscle sense of a finger oran arm as well as to transfer the tactile effect through direct contact.Optionally, two or more haptic modules 354 can be provided to the mobiledevice 300 in accordance with a configuration type of the mobile device300.

The memory 360 may store a program for an operation of the controller380, or may temporarily store inputted/outputted data (e.g., phonebook,message, still image, video, etc.). And, the memory 360 may store dataof vibrations and sounds of various patterns outputted in response to atouch input to the touchscreen.

The memory 360 may be implemented using any type or combination ofsuitable volatile and non-volatile memory or storage devices, includinghard disk, random access memory (RAM), static random access memory(SRAM), electrically erasable programmable read-only memory (EEPROM),erasable programmable read-only memory (EPROM), programmable read-onlymemory (PROM), read-only memory (ROM), magnetic memory, flash memory,magnetic or optical disk, multimedia card micro type memory, card-typememory (e.g., SD memory or XD memory), or other similar memory or datastorage device. Furthermore, the mobile device 300 is able to operate inassociation with the web storage for performing a storage function ofthe memory 360 on the Internet.

The interface unit 370 may play a role as a passage to every externaldevice connected to the mobile device 300 with external devices. Theinterface unit 370 receives data from the external devices, delivers asupplied power to the respective elements of the mobile device 300, orenables data within the mobile device 300 to be transferred to theexternal devices. For instance, the interface unit 370 may include awired/wireless headset port, an external charger port, a wired/wirelessdata port, a memory card port, a port for coupling to a device having anidentity module, audio input/output ports, video input/output ports, anearphone port, and the like.

The identity module is a chip for storing various kinds of informationfor authenticating a use authority of the mobile device 300 and mayinclude User Identify Module (UIM), Subscriber Identity Module (SIM),Universal Subscriber Identity Module (USIM), and the like. A devicehaving the identity module (hereinafter called ‘identity device’) can bemanufactured in form of a smart card. Therefore, the identity device isconnectible to the mobile device 300 through a port.

When the mobile device 300 is connected to an external cradle, theinterface unit 370 becomes a passage for supplying the mobile device 300with a power from the cradle or a passage for delivering various commandsignals input from the cradle by a user to the mobile device 300. Eachof the various command signals inputted from the cradle or the power canoperate as a signal for recognizing that the mobile device 300 iscorrectly installed in the cradle.

The controller 380 typically controls the overall operations of themobile device 300. For example, the controller 380 performs the controland processing associated with voice calls, data communications, videocalls, and the like. The controller 380 may include a multimedia module381 that provides multimedia playback. The multimedia module 381 may beconfigured as a part of the controller 380, or implemented as a separatecomponent. Moreover, the controller 380 is able to perform a patternrecognition processing for recognizing a writing input and a picturedrawing input performed on the touchscreen as a text and an image,respectively.

The power supply unit 390 is supplied with an external or internal powerand then supplies a power required for an operation of each component,under the control of the controller 380.

Various embodiments described herein may be implemented in a recordingmedium readable by a computer or a device similar to the computer usingsoftware, hardware, or some combination thereof for example.

For hardware implementation, the embodiments described herein may beimplemented within at least one of application specific integratedcircuits (ASICs), digital signal processors (DSPs), digital signalprocessing devices (DSPDs), programmable logic devices (PLDs), fieldprogrammable gate arrays (FPGAs), processors, controllers,micro-controllers, microprocessors, other electronic units designed toperform the functions described herein, and a selective combinationthereof. Such embodiments may also be implemented by the controller 180.

For software implementation, the embodiments described herein may beimplemented with separate software modules, such as procedures andfunctions, each of which performs one or more of the functions andoperations described herein. The software codes can be implemented witha software application written in any suitable programming language andmay be stored in memory such as the memory 360, and executed by acontroller or processor, such as the controller 380.

FIG. 4 is a block diagram showing a digital device according to anotherembodiment of the present invention.

Another example of a digital device 400 may include a broadcastreceiving unit 405, an external device interface 435, a storage unit440, a user input interface 450, a controller 470, a display unit 480,an audio output unit 485, a power supply unit 490, and a photographingunit (not shown). The broadcast receiving unit305 may include at leastone of one or more tuner 410, a demodulator 420, and a network interface430. Yet, in some cases, the broadcast receiving unit 405 may includethe tuner 410 and the demodulator 420 without the network interface 430,or may include the network interface 430 without the tuner 410 and thedemodulator 420. The broadcast receiving unit 405 may include amultiplexer (not shown) to multiplex a signal, which is subjected to thetuner 410 and demodulated by the demodulator 420, and a signal receivedthrough the network interface 40. In addition, the broadcast receivingunit 405 can include a demultiplexer (not shown) and demultiplex amultiplexed signal, a demodulated signal, or a signal received throughthe network interface 430.

The tuner 410 may receive a radio frequency (RF) broadcast signal bytuning in to a channel selected by the user or all previously storedchannels among RF broadcast signals received through an antenna. And,the tuner 410 converts the received RF broadcast signal into an IF(intermediate frequency) signal or a baseband signal.

For instance, if a received RF broadcast signal is a digital broadcastsignal, it is converted into a digital IF (DIF) signal. If a received RFbroadcast signal is an analog signal, it is converted into an analogbaseband video/audio signal (CVBS/SIF). Namely, the tuner 410 is able toprocess both of the digital broadcast signal and the analog signal. Theanalog baseband video/audio signal (CVBS/SIF) outputted from the tuner410 may be directly inputted to the controller 470.

The tuner 410 may receive an RF broadcast signal of a single carrier ormultiple carriers. The tuner 410 sequentially tunes in to and receivesRF broadcast signals of all broadcast channels stored through thechannel memory function among RF broadcast signals received through theantenna and is then able to convert it into an intermedia frequencysignal or a baseband signal (DIF: digital intermediate frequency orbaseband signal).

The demodulator 420 receives and demodulates the digital IF signal (DIF)converted by the tuner 410 and is then able to channel decoding and thelike. To this end, the demodulator 420 may include a Trellis decoder, ade-interleaver, a Reed-Solomon decoder and the like, or may include aconvolution decoder, a de-interleaver, a Reed-Solomon decoder and thelike.

The demodulator performs demodulation and channel decoding and is thenable to output a stream signal TS. In this case, the stream signal mayinclude a signal of multiplexing a video signal, an audio signal and/ora data signal. For instance, the stream signal may include MPEG-2TS(transport stream) in which a video signal of PMEG-2 and an audio signalof Dolby AC-3 are multiplexed.

The stream signal outputted from the demodulator 420 may be inputted tothe controller 470. The controller 470 can control demultiplexing,audio/video signal processing, etc. Furthermore, the controller 470 cancontrol outputs of video and audio through the display 480 and o theaudio output unit 485, respectively.

The external device interface 435 may provide an interfacing environmentbetween the digital device 300 and various external devices. To thisend, the external device interface 435 may include an A/V input/outputunit (not shown) or a wireless communication unit (not shown).

The external device interface 435 can be connected with external devicessuch as a digital versatile disk (DVD), a Blu-ray player, a game device,a camera, a camcorder, a computer (notebook computer), a tablet PC, asmartphone, a cloud and the like by wire/wireless. The external deviceinterface 435 delivers a signal containing data such as an image, avideo, an audio and the like, which is inputted through the connectedexternal device, to the controller 470 of the digital device. Thecontroller 470 may control a data signal of the processed image, videoand audio and the like to be outputted to the connected external device.To this end, the external device interface 435 may further include anA/V input/output unit (not shown) or a wireless communication unit (notshown).

In order to input video and audio signals of an external device to thedigital device 400, the A/V input/output unit may include a USB(Universal Serial Bus) terminal, a composite video banking sync (CVBS)terminal, a component terminal, an S-video terminal (analog), a digitalvisual interface (DVI) terminal, a high definition multimedia interface(HDMI) terminal, an RGB terminal, a D-SUB terminal, etc.

The wireless communication unit can perform short-range wirelesscommunication with another digital device. The digital device 400 may benetworked with other digital devices by communication protocols such asBluetooth, radio frequency identification (RFID), infrared dataassociation (IrDA), ultra wideband (UWB), ZigBee, and digital livingnetwork alliance (DLNA), etc. for example.

The external device interface 435 may perform input/output operationswith a set-top box (STB) by being connected thereto through at least oneof the aforementioned terminals.

Meanwhile, the external device interface 435 may receive an applicationor an application list within an adjacent external device and thenforward it to the controller 470 or the storage unit 440.

The network interface 430 may provide an interface for connecting thedigital device 400 to wired/wireless networks including Internetnetwork. The network interface 430 may have Ethernet terminal and thelike for an access to a wired network for example. For the access to thewireless network, the network interface 430 may use communicationspecifications such as WLAN (Wireless LAN) (Wi-Fi), Wibro (Wirelessbroadband), Wimax (World Interoperability for Microwave Access), HSDPA(High Speed Downlink Packet Access), etc.

The network interface 430 may transceive data with another user oranother digital device through the accessed network or another networklinked to the accessed network. Particularly, the network interface 430may send a portion of the content data stored in the digital device 400to a user/digital device selected from other users/digital devicespreviously registered at the digital device 400.

Meanwhile, the network interface 430 may access a prescribed webpagethrough the accessed network or another network linked to the accessednetwork. Namely, the network interface 430 accesses a prescribed webpagethrough a network and is then able to transceive data with acorresponding server. Besides, the network interface 430 can receivecontents or data provided by a content provider or a network operator.Namely, the network interface 430 may receive contents (e.g., movie,advertisement, game, VOD, broadcast signal, etc.) provided by thecontent provider or a network provider and information associated withthe contents through the network. The network interface 430 may receiveupdate information and file of firmware provided by the networkoperator. And, the network interface 430 may send data to the internetor content provider or the network operator.

Moreover, the network interface 430 may select a desired applicationfrom open applications and receive it through a network.

The storage unit 440 may store programs for various signal processingand controls within the controller 470, and may also store a processedvideo, audio or data signal.

In addition, the storage unit 440 may execute a function of temporarilystoring a video, audio or data signal inputted from the external deviceinterface 435 or the network interface 430. The storage unit 440 maystore information on a prescribed broadcast channel through a channelmemory function.

The storage unit 440 may store an application or an application listinputted from the external device interface 435 or the network interface430.

And, the storage unit 440 may store various platforms which will bedescribed later.

The storage unit 440 may include storage media of one or more types,such as a flash memory type, a hard disk type, a multimedia card microtype, a card type memory (e.g. SD or XD memory), RAM, EEPROM, etc. Thedigital device 400 may play content files (a video file, a still imagefile, a music file, a text file, an application file, etc.) stored inthe storage unit 440 and provide them to the user.

FIG. 4 illustrates an embodiment in which the storage unit 440 isseparated from the controller 470, by which the present invention isnon-limited. In other words, the storage unit 440 may be included in thecontroller 470.

The user input interface 450 may forward a signal inputted by a user tothe controller 470 or forward a signal outputted from the controller 470to the user.

For example, the user input interface 450 may receive control signalsfor power on/off, channel selection, screen settings and the like from aremote controller 500, or transmit control signals of the controller 470to the remote controller 500, according to various communication schemessuch as RF communication, IR communication, and the like.

The user input interface 450 can forward control signals inputtedthrough a power key, a channel key, a volume key, and a local key (notshown) for a setup value or the like to the controller 470.

The user input interface 450 may forward a control signal inputted froma sensing unit (not shown) sensing a gesture of a user to the controller470 or transmit a signal of the controller 470 to the sensing unit (notshown). Here, the sensing unit (not shown) may include a touch sensor, avoice sensor, a location sensor, an action sensor, etc.

The controller 470 may generate and output a signal for a video or audiooutput by demultiplexing a stream inputted through the tuner 410, thedemodulator 420 or the external device interface 435 or processingdemultiplexed signals.

A video signal processed by the controller 470 can be inputted to thedisplay unit 380 and displayed as an image corresponding to the videosignal. In addition, the video signal video-processed by the controller470 can be inputted to an external output device through the externaldevice interface 435.

An audio signal processed by the controller 470 can be audio-outputtedto the audio output unit 485. Moreover, the audio signal processed bythe controller 470 can be inputted to the external output device throughthe external device interface 435.

The controller 470 may include a demultiplexer, an image processor, andthe like, which are not shown in FIG. 4.

The controller 470 can control the overall operations of the digitaldevice 400. For example, the controller 470 can control the tuner 410 totune in to an RF broadcast corresponding to a channel selected by a useror a previously stored channel.

The controller 470 can control the digital device 400 according to auser command input through the user input interface 450 or an internalprogram. Particularly, the controller 470 can control the digital device400 to access a network to download an application or an applicationlist desired by a user to the digital device 400.

For example, the controller 470 may control the tuner 410 to receive asignal of a channel selected in response to a prescribed channelselection command received through the user input interface 450. And,the controller 470 may process a video, audio or data signal of theselected channel. The controller 470 may control information on achannel selected by the user to be outputted together with a processedvideo or audio signal through the display unit 480 or the audio outputunit 485.

For another example, the controller 470 may control a video signal or anaudio signal, which is inputted through the external device interfaceunit 435 from an external device (e.g., a camera or a camcorder), to beoutputted through the display unit 480 or the audio output unit 485 inresponse to an external device image play command received through theuser input interface 450.

Meanwhile, the controller 470 can control the display unit 480 todisplay a video. For example, the controller 470 can control a broadcastvideo inputted through the tuner 410, an external input video inputtedthrough the external device interface 435, a video inputted through thenetwork interface 430, or a video stored in the storage unit 440 to bedisplayed on the display unit 480. Here, the video displayed on thedisplay unit 480 may include a still image or moving images or mayinclude a 2D or 3D video.

The controller 470 may control a content to be played. Here, the contentmay include a content stored in the digital device 400, a receivedbroadcast content, or a content inputted externally. The content mayinclude at least one of a broadcast video, an external input video, anaudio file, a still image, an accessed web screen, and a document file.

The controller 470 may control an application or an application list,which is located in the digital device 300 or downloadable from anexternal network, to be displayed when an application view menu isentered.

The controller 470 may control installation and execution ofapplications downloaded from an external network together with varioususer interfaces. Moreover, the controller 470 can control a videorelated to a launched application to be displayed on the display unit480 by a user's selection.

Meanwhile, a channel browsing processor (not shown) configured togenerate a thumbnail image corresponding to a channel signal or anexternal input signal may be further included.

The channel browsing processor may receive an input of a stream signal(TS) outputted from the demodulator 420 or an input of a stream signaloutputted from the external device interface 435, extract a video fromthe inputted stream signal, and then generate a thumbnail image. Thegenerated thumbnail image can be directly inputted to the controller 470or may be inputted to the controller 470 by being encoded. Moreover, thegenerated thumbnail image may be encoded into a stream and then inputtedto the controller 470. The controller 470 may display a thumbnail listincluding a plurality of thumbnail images on the display unit 480 usingthe inputted thumbnail images. The thumbnail images included in thethumbnail list can be updated sequentially or simultaneously.Accordingly, the user can conveniently check content of a plurality ofbroadcast channels.

The display unit 480 may convert each of a video signal, a data signal,and an OSD signal processed by the controller 470 or each of a videosignal and a data signal received from the external device interface 435into R, G and B signals to generate a drive signals.

The display unit 480 may include a PDP, an LCD, an OLED, a flexibledisplay, a 3D display, or the like.

The display unit 480 may be configured as a touchscreen and used as aninput device as well as an output device.

The audio output unit 485 receives a signal audio-processed by thecontroller 470, for example, a stereo signal, a 3.1 channel signal or a5.1 channel signal, and then outputs the received signal as audio. Theaudio output unit 485 may be configured as one of speakers of varioustypes.

Meanwhile, the digital device 400 may further include the sensing unit(not shown) for sensing a gesture of the user, which includes at leastone of a touch sensor, a voice sensor, a location sensor, and an actionsensor, as described above. A signal sensed by the sensing unit (notshown) can be delivered to the controller 470 through the user inputinterface 450.

The digital device 400 may further include a photographing unit (notshown) for photographing a user. Image information acquired by thephotographing unit (not shown) can be inputted to the controller 470.

The controller 470 may sense a gesture of a user from an image capturedby the photographing unit (not shown) or a signal sensed by the sensingunit (not shown), or by combining the image and the signal.

The power supply unit 490 may supply a corresponding power to thedigital device 400 overall.

Particularly, the power supply unit 490 can supply the power to thecontroller 470 configurable as a system-on-chip (SoC), the display unit480 for a video display, and the audio output unit 485 for an audiooutput.

To this end, the power supply unit 490 may include a converter (notshown) configured to convert an AC power to a DC power. Meanwhile, forexample, if the display unit 480 is configured as an LCD panel having amultitude of backlight lamps, the power supply unit 490 may furtherinclude an inverter (not shown) capable of PWM (pulse width modulation)operation for luminance variation or dimming drive.

The remote controller 500 sends a user input to the user input interface450. To this end, the remote controller 500 can use Bluetooth, RFcommunication, IR communication, UWB, ZigBee, etc.

In addition, the remote controller 500 can receive audio, video or datasignal outputted from the user input interface 450 and then display thereceived signal or output the same as audio or vibration.

The above-described digital device 400 may include a digital broadcastreceiver capable of processing digital broadcast signals of ATSC or DVBof a stationary or mobile type.

Regarding the digital device according to the present invention, some ofthe illustrated components may be omitted or new components (not shown)may be further added as required. On the other hand, the digital devicemay not include the tuner and the demodulator, differently from theaforementioned digital device, and may play a content by receiving thecontent through the network interface or the external device interface.

FIG. 5 is a block diagram showing the detailed configuration of each ofcontrollers of FIGS. 2 to 4 according to one embodiment of the presentinvention.

One example of the controller may include a demultiplexer 510, a videoprocessor 520, an OSD generator 540, a mixer 550, a frame rate converter(FRC) 555, and a formatter 560. Besides, the controller may furtherinclude an audio processor and a data processor (not shown).

The demultiplexer 510 demultiplexes an inputted stream. For instance,the demultiplexer 510 can demultiplex an inputted stream signal into anMPEG-2 TS video, audio and data signals. Herein, the stream signalinputted to the demultiplexer may include a stream signal outputted fromthe tuner, demodulator or external device interface.

The video processor 520 performs a video processing of the demultiplexedvideo signal. To this end, the video processor 520 may include a videodecoder 525 and a scaler 535.

The video decoder 525 can decode the demultiplexed video signal, and thescaler 535 can scale the resolution of the decoded video signal to beoutputtable from the display.

The video decoder 525 can support various specifications. For instance,the video decoder 525 performs a function of MPEG-2 decoder if a videosignal is encoded by MPEG-2. And, the video decoder 535 performs afunction of H.264 decoder if a video signal is encoded by DMB (digitalmultimedia broadcasting) or H.264 .

Meanwhile, the video signal decoded by the image processor 520 isinputted to the mixer 550.

The OSD generator 540 may generate OSD data according to a user input orby itself. For example, the OSD generator 540 may generate data to bedisplayed on the screen of the display 380 in the graphic or text formon the basis of a control signal of a user input interface. Thegenerated OSD data may include various data such as a user interfacescreen of the digital device, various menu screens, widgets, icons,viewing rate information and the like. The OSD generator 540 cangenerate data to display a caption of a broadcast video or EPG basedbroadcast information.

The mixer 550 mixes the OSD data generated by the OSD generator 540 andthe video signal processed by the video processor 520. The mixer 550then provides the mixed signal to the formatter 560. By mixing thedecoded video signal and the OSD data, OSD is displayed in a manner ofoverlaying a broadcast video or an external input video.

The frame rate converter (FRC) 555 may convert a frame rate of aninputted video. For example, the frame rate converter 555 can convertthe frame rate of an inputted 60 Hz video into a frame rate of 120Hz or240Hz according to an output frequency of the display unit. As describedabove, there may exist various methods of converting a frame rate. Forinstance, in case of converting a frame rate into 120 HZ from 60 Hz, theframe rate converter 555 can perform the conversion by inserting a firstframe between the first frame and a second frame or inserting a thirdframe predicted from the first and second frames. For another instance,in case of converting a frame rate into 240 Hz from 60 Hz, the framerate converter 555 can perform the conversion by further inserting threesame or predicted frames between the existing frames. Meanwhile, in caseof not performing a separate frame conversion, the frame rate converter555 may be bypassed.

The formatter 560 may change the output of the frame rate converter 555,which is inputted thereto, to fit an output format of the display unit.For example, the formatter 560 can output an RGB data signal. In thiscase, this RGB data signal can be outputted as low voltage differentialsignal (LVDS) or mini-LVDS. If an inputted output of the frame rateconverter 555 is a 3D video signal, the formatter 560 outputs the signalby configuring a 3D format to fit the output format of the display unit,whereby a 3D service can be supported through the display unit.

Meanwhile, an audio processor (not shown) in the controller can performaudio processing of a demultiplexed audio signal. Such an audioprocessor (not shown) can provide supports to process various audioformats. For instance, if an audio signal is encoded in format ofMPEG-2, MPEG-4, AAC, HE-AAC, AC-3, BSAC, or the like, a correspondingdecoder is further included to process the audio signal.

And, the audio processor (not shown) in the controller can process base,treble, volume adjustment and the like.

A data processor (not shown) in the controller can process ademultiplexed data signal. For example, when a demultiplexed data signalis encoded, the data processor can decode the encoded demultiplexed datasignal. Here, the encoded data signal may be EPG information includingbroadcast information such as start and end times of a broadcast programbroadcasted on each channel, and the like.

Meanwhile, the above-described digital device is one example accordingto the present invention. And, at least one of the components may beintegrated, added or omitted depending on options of an actuallyembodied digital device. In particular, if necessary, at least two ormore components can be integrated into a single component or aprescribed component can be divided into at least two or morecomponents. Moreover, a function performed by each block is provided todescribe one embodiment of the present invention. A detailed operationor device for the function may non-limit the scope of the appendedclaims and their equivalents of the present invention.

Meanwhile, a digital device may include an image signal processingdevice configured to process a signal of an image saved in thecorresponding device or a signal of an inputted image. Examples of theimage signal processing device may include a settop box (STB) failing toinclude the display unit 480 and the audio output unit 485 shown in FIG.4, the aforementioned DVD player, a Blu-ray player, a game device, acomputer and the like.

FIG. 6 is a diagram of an input means connected to each of the digitaldevices shown in FIGS. 2 to 4 according to one embodiment of the presentinvention.

In order to control a digital device 600, a front panel (not shown inthe drawing) or a control means (e.g., an input means) installed in thedigital device 600 is used.

Meanwhile, as a user interface device (UID) capable of a wire/wirelesscommunication, the control means includes a remote controller 610, a keyboard 630, a pointing device 620, a touchpad, or the like, mainlyembodied for the purpose of controlling the digital device 600. And, acontrol means dedicated to an external input by being connected to thedigital device 600 may be included as well. Besides, the control meansmay further include a mobile device (e.g., a smartphone, a tablet PC,etc.) capable of controlling the digital device 600 through a modeswitching or the like despite not having the purpose of controlling thedigital device 600. For clarity, a pointing device is taken as oneexample for the description in the present specification, by which thepresent invention is non-limited.

The input means can communicate with the digital device by employing atleast one of communication protocols as necessary. In this case, thecommunication protocols may include Bluetooth, RFID (Radio FrequencyIdentification), IrDA (infrared Data Association), UWB (Ultra Wideband),ZigBee, DLNA (Digital Living Network Alliance), RS and the like.

The remote controller 610 is a general input means provided with variouskey buttons required for controlling the digital device 600.

The pointing device 620 provided with a gyro sensor and the likedelivers a prescribed control command to the digital device 600 byembodying a corresponding pointer on a screen of the digital device 600based on a user's motion, a pressure, a rotation and the like. Thepointing device 620 may be called one of various names such as a magicremote controller, a magic controller and the like.

As the digital device 600 is an intelligence integrated digital devicecapable of providing various services such as a web browser, anapplication, an SNS (social network service) and the like as well asbroadcasts, it is difficult to control the digital device 600 using aconventional remote controller 610. Hence, the keyboard 630 is embodiedinto a configuration similar to a PC keyboard to facilitate inputs oftext and the like by complementing the control difficulty.

Meanwhile, the control means such as the remote controller 610, thepointing device 620, the keyboard 630, or the like is provided with atouchpad as necessary and is usable for the various control purposes offacilitating text inputs, pointer shifts, zoom-in/out of photo or video,and the like.

The digital device described in the present specification uses OS and/orWeb OS as a platform. Hereinafter, such a processing as a WebOS basedconfiguration or algorithm may be performed by the controller of theabove-described digital device and the like. In this case, thecontroller is used in a broad sense including the controllers shown inFIGS. 2 to 5. Hence, in the following description, regarding aconfiguration for processing WebOS based or related services,applications, contents and the like in a digital device, hardware orcomponent including software, firmware and the like is named acontroller.

Such a Web OS based platform may improve development independency andfunctional extensibility by integrating services, applications and thelike based on Luna-service Bus for example and is able to increaseapplication development productivity based on a web applicationframework. In addition, system resources and the like are efficientlyused through a WebOS process and resource management, wherebymultitasking can be supported.

Meanwhile, a Web OS platform described in the present specification maybe available not only for stationary devices such as personal computers(PCs), TVs and settop boxes (STBs) but also for mobile devices such ascellular phones, smartphones, tablet PCs, laptops, wearable devices, andthe like.

A software structure for a digital device is a monolithic structurecapable of solving conventional problems depending on markets and hasdifficulty in external application with a multi-threading based signalprocess and closed product. In pursuit of new platform baseddevelopment, cost innovation through chipset replacement and UIapplication and external application development efficiency, layeringand componentization are performed to obtain a 3-layered structure andan add-on structure for an add-on, a single source product and an openapplication. Recently, modular design of a software structure has beenconducted in order to provide a web open application programminginterface (API) for an echo system and modular architecture of afunctional unit or a native open API for a game engine, and thus amulti-process structure based on a service structure has been produced.

FIG. 7 is a diagram showing Web OS architecture according to oneembodiment of the present invention.

The architecture of Web OS platform is described with reference to FIG.7 as follows.

The platform can be mainly classified into a system library based Web OScore platform, an application, a service and the like.

The architecture of the Web OS platform includes a layered structure.OS, system library(s), and applications exist in a lowest layer, a nextlayer and a most upper layer, respectively.

First of all, regarding the lowest layer, as a Linux kernel is includedas an OS layer, Linux may be included as an OS of the digital device.

Above the OS layer, BSP/HAL (Board Support Package/Hardware Abstractionlayer, Web OS core modules layer, service layer, Luna-Service Bus layer,Enyo framework/NDK(Native Developer's Kit)/QT layer, and an applicationlayer (as a most upper layer) exist in order.

Meanwhile, some layers can be omitted from the aforementioned Web OSlayer structure. A plurality of layers can be integrated into a singlelayer, and vice versa.

The Web OS core module layer may include LSM (Luna Surface Manager) formanaging a surface window and the like, SAM (System & ApplicationManage) for managing launch, running state and the like of anapplication, WAM (Web Application Manager) for managing Web applicationand the like based on WebKit, etc.

The LSM manages an application window appearing on a screen. The LSM isin charge of a display hardware (HW), provides a buffer capable ofrendering substance required for applications, and outputs a compositionof rendering results of a plurality of application to a screen.

The SAM manages a performance policy per conditions of system andapplication.

Meanwhile, since Web OS may regard a web application (Web App) as abasic application, the WAM is based on Enyo Framework.

A service use of application is performed through Luna-service Bus. Anew service may be registered as the Bus, and an application can findand use a service required for itself.

The service layer may include services of various service levels such asTV service, Web OS service and the like. Meanwhile, the Web OS servicemay include a media server, a Node.JS and the like. Particularly,Node.JS service supports javascript for example.

The Web OS service is Linux process of implementing a function logic andcan communicate through Bus. This can be mainly divided into four partsand is constructed with a TV process, services migrating into Web OSfrom an existing TV or services corresponding tomanufacturer-differentiated services, Web OS common service, and Node.jsservice developed with javascript and used through Node.js.

The application layer may include all applications supportable by thedigital device, e.g., TV application, showcase application, nativeapplication Web application, etc.

Application on Web OS may be sorted into Web Application, PDK (PalmDevelopment Kit) application, QML (Qt Meta Language or Qt ModelingLanguage) application and the like according to implementing methods.

The Web Application is based on WebKit engine and is run on WAM Runtime.Such a web application is based on Enyo Framework or may be run in amanner of being developed based on general HTML5, CSS (cascading stylesheets), and javascript.

The PDK application includes a native application and the like developedwith C/C++ based on PDK provided for a 3^(rd) party or an externaldeveloper. The PDK means a set of development libraries and toolsprovided to enable a third party (e.g., a game, etc.) to develop anative application (C/C++). The PDK application can be used to developan application of which performance is significant.

The QML application is a Qt based native application and includes basicapplications (e.g., card view, home dashboard, virtual keyboard, etc.)provided with Web OS platform. Herein, QML is a mark-up language of ascript type instead of C++.

Meanwhile, in the above description, the native application means anapplication that is developed with C/C++, complied, and run in binaryform. Such a native application has an advantage of a fast runningspeed.

FIG. 8 is a diagram showing an architecture of Web OS device accordingto one embodiment of the present invention.

FIG. 8 is a block diagram based on a runtime of a Web OS device, whichcan be understood with reference to the layered structure shown in FIG.7.

The following description is made with reference to FIG. 7 and FIG. 8.

Referring to FIG. 8, above a system OS (Linux) and system libraries,services, applications and Web OS core modules are included. And,communications among them can be performed through Luna-Service-Bus.

Node.js services (e-mail, contact, calendar, etc.) based on HTML5, CSS,and java script, Web OS services such as Logging, backup, file notify,database (DB), activity manager, system policy, AudioD (Audio Daemon),update, media server and the like, TV services such as EPG (ElectronicProgram Guide), PVR (Personal Video Recorder), data broadcasting and thelike, CP services such as voice recognition, Now on, Notification,search, ACR (Auto Content Recognition), CBOX (Contents List Browser),wfdd, DMR, Remote Application, download, SDPIF (Sony Philips DigitalInterface Format) and the like, native applications such as PDKapplications, browser, QML application and the like, and Enyo Frameworkbased UI related TV applications and Web applications are processedthrough the Web OS core module like the aforementioned SAM, WAM and LSMvia Luna-Service-Bus. Meanwhile, in the above description, it is notmandatory for the TV applications and the Web applications to beEnyo-Framework-based or UI-related.

CBOX can manage a list and metadata for contents of such an externaldevice connected to TV as USB, DLNA, Cloud and the like. Meanwhile, theCBOX can output a content listing of various content containers such asUSB, DMS, DVR, Cloud and the like in form of an integrated view. And,the CBOX shows a content listing of various types such as picture,music, video and the like and is able to manage the correspondingmetadata. Besides, the CBOX can output a content of an attached storageby real time. For instance, if a storage device such as USB is pluggedin, the CBOX should be able to output a content list of thecorresponding storage device. In doing so, a standardized method for thecontent list processing may be defined. And, the CBOX may accommodatevarious connecting protocols.

SAM is provided to enhance improvement and extensibility of modulecomplexity. Namely, for instance, since an existing system managerhandles various functions (e.g., system UI, window management, webapplication run time, constraint condition processing on UX, etc.) by asingle process, implementation complexity is very high. Hence, byseparating major functions and clarifying an inter-function interface,implementation complexity can be lowered.

LSM supports system UX implementation (e.g., card view, launcher, etc.)to be independently developed and integrated and also supports thesystem UX implementation to easily cope with a product requirementchange and the like. In case of synthesizing a plurality of applicationscreens like App On App, the LSM enables multitasking by utilizinghardware (HW) resource to the maximum, and is able to provide a windowmanagement mechanism for multi-window, 21:9 and the like.

LSM supports implementation of system UI based on QML and enhancesdevelopment productivity thereof. QML UX can easily configure a screenlayout and a UI component view and facilitates development of a code forprocessing a user input. Meanwhile, an interface between QML and Web OScomponent is achieved through QML extensive plug-in, and a graphicoperation of application may be based on wayland protocol, luna-servicecall and the like.

LSM is an abbreviation of Luna Surface Manager, as described above, andperforms a function of an application window compositor.

LSM synthesizes an independently developed application, a US componentand the like and then outputs the synthesized one to a screen. Withrespect to this, if components such as Recents application, showcaseapplication, launcher application and the like render contents of theirown, respectively, LSM defines an output region, an interoperatingmethod and the like as a compositor. So to speak, the LSM (i.e.,compositor) processes graphic synthesis, focus management, input eventand the like. In doing so, LSM receives an event, a focus and the likefrom an input manager. Such an input manager may include a remotecontroller, an HID (e.g., mouse & keyboard), a joy stick, a game pad, anapplication remote, a pen touch and the like.

Thus, LSM supports a multiple window model and can be simultaneously runon all applications owing to system UI features. With respect to this,LSM can support launcher, recents, setting, notification, systemkeyboard, volume UI, search, finger gesture, Voice Recognition (STT(Sound to Text), TTS (Text to Sound), NLP (Natural Language Processing),etc.), pattern gesture (camera, MRCU (Mobile Radio Control Unit)), Livemenu, ACR (Auto Content Recognition), and the like.

FIG. 9 is a diagram showing a graphic composition flow in a Web OSdevice according to one embodiment of the present invention.

Referring to FIG. 9, a graphic composition processing can be performedthrough a web application manager 910 in charge of a UI process, awebkit 920 in charge of a web process, an LSM 930, and a graphic manager(GM) 940.

If a web application based graphic data (or application) is generated asa UI process from the web application manager 910, the generated graphicdata is forwarded to a full-screen application or the LSM 930.Meanwhile, the web application manager 910 receives an applicationgenerated from the webkit 920 for sharing the GPU (graphic processingunit) memory for the graphic managing between the UI process and the webprocess and then forwards it to the LSM 930 if the application is notthe full-screen application. If the application is the full-screenapplication, it can bypass the LSM 930. In this case, it may be directlyforwarded to the graphic manager 940.

The LSM 930 sends the received UI application to a wayland compositorvia a wayland surface. The wayland compositor appropriately processes itand then forwards it to the graphic manager. Thus, the graphic dataforwarded by the LSM 930 is forwarded to the graphic manager compositorvia the LSM GM surface of the graphic manager 940 for example.

Meanwhile, as described above, the full-screen application is directlyforwarded to the graphic manager 940 without passing through the LSM930. Such an application is processed by the graphic manager compositorvia the WAM GM surface.

The graphic manager processes all graphic data within the Web OS device.The graphic manager receives all the graphic data through the GM surfacelike data broadcasting application, caption application and the like aswell as the data through the LSM GM and the data through the WAM GMsurface and then processes them to be outputted to the screenappropriately. Herein, a function of the GM compositor is equal orsimilar to that of the aforementioned compositor.

FIG. 10 is a diagram showing a media server according to one embodimentof the present invention. FIG. 11 is a block diagram showing aconfiguration of a media server according to one embodiment of thepresent invention. FIG. 12 is a diagram showing the relation between amedia server and according to one embodiment of the present inventionand a TV service.

A media server supports executions of various multimedia in a digitaldevice and manages necessary resources. The media server can efficientlyuse a hardware resource required for a media play. For instance, themedia server needs audio/video hardware resource to execute multimedia,and is able to efficiently utilize the resource by managing a currentresource use status. Generally, a stationary (or standing) device havinga screen larger than that of a mobile device requires more hardwareresources on multimedia execution and needs a faster encoding/decodingand graphic data transfer speed due to a massive data size. Meanwhile,the media server should be able to handle abroadcasting/recording/tuning task, a task of recording at the same timeof viewing, a task of displaying both a sender screen and a receiverscreen during a video call, and the like as well as a streaming and afile based play. Yet, since hardware resources such as an encoder, adecoder, a tuner, a display engine, and the like are limited by chipsetunits, it is difficult for the media server to execute several tasks atthe same time. Hence, the media server handles the tasks in a manner ofrestricting a use scenario or receiving an input of user selection.

The media server can add robustness to system stability. For instance,by removing an erroneous play pipeline per pipeline in the course of amedia play and then re-maneuvering the media play, another media play isnot affected even if such an error occurs. Such a pipeline is a chain ofconnecting the respective unit functions (e.g., decoding, analysis,output, etc.) in case of a media play request, and necessary unitfunctions may be changed according to a media type and the like.

The media server may have extensibility. For instance, the media servercan add a pipeline of a new type without affecting an existingimplementation scheme. For instance, the media server can accommodate acamera pipeline, a video conference (Skype) pipeline, a third-partypipeline and the like.

The media server can handle a general media play and a TV task executionas separate services, respectively. The reason for this is that aninterface of a TV service is different from a media play case. In theabove description, the media server supports operations of ‘setchannel’,‘channelup’, ‘channeldown’, ‘channeltuning’, ‘recordstart’ and the likein association with the TV service but supports operations of ‘play’,‘pause’, ‘stop’ and the like in association with the general media play,thereby supporting different operations for the two services,respectively. Thus, the media server is able to handle the servicesseparately.

The media server may control or manage resource management functionsintegratedly. Hardware resource allocation, recovery and the like in adevice are integratedly performed in the media server. Particularly, aTV service process delivers a currently running task, a current resourceallocation status and the like to the media server. Each time each mediais executed, the media server secures a resource, activates a pipeline,and performs a grant of execution by a priority (e.g., policy), aresource recovery of other pipelines and the like in response to a mediaexecution request based on a current resource status occupied by eachpipeline. Herein, a predefined execution priority and a necessaryresource information for a specific request are managed by a policymanager, and a resource manager can handle resource allocation, recoveryand the like by communicating with the policy manager.

The media server can retain an ID (identifier) for every operationrelated to a play. For instance, based on an identifier, the mediaserver can give a command by indicating a specific pipeline. For two ormore media plays, the media server may give a command to pipelines bydistinguishing the two from each other.

The media server may be in charge of a play of HTMS 5 standard media.

Besides, the media server may follow a TV reconfiguration range for aseparate service processing of a TV pipeline. The media server can bedesigned irrespective of the TV reconfiguration range. If the TV is notseparately service-processed, when a problem arises from a specifictask, the TV may be re-executed entirely.

The media server is so-called uMS, i.e., a micro media server. Herein, amedia player is a media client. This may mean a webkit for HTML 5 videotag, camera, TV, Skype, 2^(nd) screen and the like.

A core function of the media server is the management of a microresource such as a resource manager, a policy manager or the like. Withrespect to this, the media server controls a playback control role on aweb standard media content. Regarding this, the media server may managea pipeline controller resource.

Such a media server supports extensibility, reliability, efficientresource usage and the like for example.

So to speak, the uMS, i.e., the media server manages and controls theuse of resources for an appropriate processing in a Web OS device suchas a resource (e.g., cloud game, MVPD (pay service, etc.), camerapreview, 2nd screen, Skype, etc.), a TV resource and the like overall,thereby functioning in managing and controlling an efficient usage.Meanwhile, when resources are used, each resource uses a pipeline forexample. And, the media server can manage and control generation,deletion, usage and the like of the pipeline for resource managementoverall.

Herein, a pipeline may be generated if a media related to a task startsto continue a job such as a parsing of request, decoding stream, videooutput, or the like. For instance, in association with a TV service orapplication, watching, recording, channel tuning or the like isindividually processed in a manner that a resource usage or the like iscontrolled through a pipeline generated in response to a correspondingrequest.

A processing structure of a media server and the like are described indetail with reference to FIG. 10 as follows.

In FIG. 10, an application or service is connected to a media server1020 through a luna-service bus 1010. The media server 1020 is connectedto generated pipelines through the luna-service bus 1010 again andmanages them.

The application or service is provided with various clients according toits property and is able to exchange data with the media server 1020 orthe pipelines through them.

The clients may include a uMedia client (webkit) for the connection tothe media server 1020, an RM (resource manager) client (C/C++) and thelike for example.

The application including the uMedia client, as described above, isconnected to the media server 1020. In particular, the uMedia clientcorresponds to a video object to be described later. Such a client usesthe media server 1020 for an operation of a video in response to arequest or the like.

Herein, the video operation relates to a video status. Loading,unloading, play (or, playback, reproduce), pause, stop and the like mayinclude all status data related to video operations. Each operation orstatus of a video can be processed through individual pipelinegeneration. Hence, the uMedia client sends status data related to thevideo operation to the pipeline manager 1022 in the media server.

The pipeline manager 1022 obtains information on a current resource of adevice through data communication with the resource manager 1024 andmakes a request for allocation of a resource corresponding to the statusdata of the uMedia client. In doing so, the pipeline manager 1022 or theresource manager 1024 controls the resource allocation through the datacommunication with the policy manager 1026 if necessary in associationwith the resource allocation and the like. For instance, if a resourceto be allocated by the resource manager in response to the request madeby the pipeline manager 1022 does not exist or is insufficient, anappropriate resource allocation or the like according to the request canbe performed according to priority comparison of the policy manager 1026and the like.

Meanwhile, the pipeline manager 1022 makes a request for pipelinegeneration for an operation according to the uMedia client's request forthe resource allocated according to the resource allocation of theresource manager 1024 to a media pipeline controller 1028.

The media pipeline controller 1028 generates a necessary pipeline underthe control of the pipeline manager 1022. Regarding the generatedpipelines, as shown in the drawing, pipelines related to play, pause,stop and the like can be generated as well as a media pipeline and acamera pipeline. Meanwhile, the pipelines may include pipelines forHTML5, Web CP, smartshare play, thumbnail extraction, NDK, cinema, MHEG(Multimedia and Hypermedia Information coding Experts Group) and thelike.

Besides, pipelines may include a service based pipeline (self-pipeline)and a URI based pipeline (media pipeline) for example.

Referring to FIG. 10, the application or service including the RM clientmay not be directly connected to the media server 1020. The reason forthis is that the application or service may directly process a media. Soto speak, in case that the application or service directly processesmedia, the media server can be bypassed. Yet, in doing so, sinceresource management is necessary for the pipeline generation and usage,a uMS connector functions for it. Meanwhile, if a resource managementrequest for the direct media processing of the application or service isreceived, the uMS connector communicates with the media server 1020including the resource manager 1024. To this end, the media server 1020should be provided with a uMS connector as well.

Hence, by receiving the resource management of the resource manager 1024through the uMS connector, the application or service can cope with therequest of the RM client. Such an RM client may process services such asnative CP, TV service, 2^(nd) screen, flash player, U-tube MSE (mediasource extensions), cloud game, Skype and the like. In this case, asdescribed above, the resource manager 1024 can manage resource throughappropriate data communication with the policy manager 1026 if necessaryfor the resource management.

Meanwhile, the URI based pipeline is processed through the media server1020 instead of the case of directly processing media like the RMclient. The URI based pipelines may include player factory, Gstreamer,streaming plug-in, DRM (Digital Rights Management) plug-in pipeline andthe like.

A method of interfacing between an application and media services isdescribed as follows.

There is an interfacing method using a service on a web application.This may be a Luna Call method using PSB (palm service bridge) or amethod using Cordova. This is to extend a display with a video tag.Besides, there may be a method of using HTMS5 standard for video tag ormedia element.

And, there is a method of interfacing using a service in PDK.

Alternatively, there is a method of using a service in an existing CP.This is usable by extending plug-in of an existing platform on the basisof luna for backward compatibility.

Finally, there is an interfacing method in case of non-Web OS. In thiscase, it is able to interface by directly calling a luna bus.

Seamless change is processed by a separate module (e.g., TVWIN), whichis a process for showing a TV on a screen preferentially without Web OSand then processing seamlessly before or during Web OS booting. Since abooting time of Web OS is considerably long, it is used to provide basicfunctions of a TV service preferentially for a quick response to auser's power-on request. And, the module is a part of a TV serviceprocess and supports a seamless change capable of providing fast bootingand basic TV functions, a factory mode and the like. And, the module maybe in charge of a switching from non-Web OS mode to Web OS mode.

Referring to FIG. 11, a processing structure of a media server isillustrated.

In FIG. 11, a solid line box may indicate a process handlingconfiguration and a dotted line box may indicate an internal processingmodule in a process. A solid line arrow may include an inter-processcall, i.e., a luna service call and a dotted line arrow may indicate anotification of register/notify or a data flow.

A service, a web application or a PDK application (hereinafter‘application) is connected to various service processing configurationsthrough a luna-service bus. Through it, the application operates or anoperation of the application is controlled.

A corresponding data processing path is changed according to a type ofan application. For instance, if the application is an image datarelated to a camera sensor, it is processed by being sent to a cameraprocessor 1130. Herein, the camera processor 1130 includes a gesturemodule, a face detection module and the like and processes image data ofthe application received. Herein, in case of data requiring a usage of apipeline and the like automatically or according to a user's selection,the camera processor 1130 may process the corresponding data bygenerating the pipeline through a media server processor 1110.

Alternatively, if an application includes audio data, the correspondingaudio can be processed through an audio processor (AudioD) 1140 and anaudio module (PulseAudio) 1150. For instance, the audio processor 1140processes audio data received from the application and then sends it toan audio module 1150. In doing so, the audio processor 1140 maydetermine the processing of the audio data by including an audio policymanager. The processed audio data is processed and handled by the audiomodule 1150. Meanwhile, the application may notify data related to theaudio data processing to the audio module 1160, which may be notified tothe audio module 1160 by a related pipeline. The audio module 1150includes ALSA(Advanced Linux Sound Architecture).

Or, in case that an application includes or processes (hereinafter‘includes’) a DRM hooked content, a corresponding content data is sentto a DRM service processor 1160. The DRM service processor 1160generates the DRM hooked content data by generating a DRM instance.Meanwhile, for the processing of the DRM hooked content data, the DRMservice processor 1160 can be connected to a DRM pipeline in a mediapipeline through the luna-service bus.

A processing for a case that an application includes media data or TVservice data (e.g., broadcast data) is described as follows.

FIG. 12 is a diagram showing details of the media service processor andthe TV service processor in FIG. 11.

The following description is made with reference to FIG. 11 and FIG. 12both.

First of all, in case that an application includes TV service data, itis processed by the TV service processor 1120/1220.

Herein, the TV service processor 1120 may include at least one of aDVR/channel manager, a broadcast module, a TV pipeline manager, a TVresource manager, a data broadcast module, an audio setting module, apath manager and the like. Alternatively, the TV service processor 1220in FIG. 12 may include a TV broadcast handler, a TV broadcast interface,a service processing unit, a TV middleware (MW), a path manager, and aBSP (NetCast). Herein, the service processing unit may mean a moduleincluding a TV pipeline manager, a TV resource manager, a TV policymanager, a USM connector and the like.

In the present specification, The TV service processor may beimplemented into the configuration shown in FIG. 11 or FIG. 12 or acombination of both configurations. some of the illustrated componentsmay be omitted or new components (not shown) may be further added asrequired.

Based on attribute or type of the TV service data received from theapplication, the TV service processor 1120/1220 sends DVR or channelassociated data to the DVR/channel manager and also sends it to the TVpipeline manager to generate and process a TV pipeline. Meanwhile, ifthe attribute or type of the TV service data is a broadcast contentdata, the TV service processor 1120 generates and processes a TVpipeline through the TV pipeline manager to process the correspondingdata through the broadcast module.

Or, a json (Javascript standard object notation) file or a file composedwith c is processed by the TV broadcast handler, sent to the pipelinemanager through the TV broadcast interface, and then processed bygenerating a TV pipeline. In this case, the TV broadcast interface sendsthe data or file through the TV broadcast handler to the TV pipelinemanager on the basis of the TV service policy so that the data or filecan be referred to for the pipeline generation.

Meanwhile, the TV pipeline manager may be controlled by the TV resourcemanager when generating one or more pipelines in response to a TVpipeline generation request from the Processing module or manager in theTV service. Meanwhile, in order to request a status and allocation of aresource allocated for the TV service in response to a TV pipelinegeneration request made by the TV pipeline manager, the TV resourcemanager may be controlled by the TV policy manager and performs datacommunication with the media server processor 1110/1210 through the uMSconnector. The resource manager in the media server processor delivers astatus and a presence/non-presence of allocation of a resource for acurrent TV service in response to a request made by the TV resourcemanager. For instance, as a result of confirmation of the resourcemanager within the media server processor 1110/1210, if all resourcesfor the TV service are already allocated, it is able to notify the TVresource manager that all current resources are completely allocated. Indoing so, the resource manager in the media server processor may requestor assign TV pipeline generation for the requested TV service byremoving a prescribed TV pipeline according to a priority or prescribedreference from TV pipelines previously assigned for the TV service,together with the notification. Alternatively, according to a statusreport of the resource manager in the media server processor 1110/1210,the TV resource manager may control TV pipelines to be appropriatelyremoved, added, or established.

Meanwhile, BSP supports backward compatibility with an existing digitaldevice for example.

The above-generated TV pipelines may operate appropriately in thecorresponding processing process under the control of the path manager.The path manager may determine or control a processing path or processof pipelines by considering an operation of a pipeline generated by themedia server processor 1110/1210 as well as the TV pipeline in theprocessing process.

If the application includes media data instead of TV service data, thedata is processed by the media server processor 1110/1210. Herein, themedia server processor 1110/1210 includes a resource manager, a policymanager, a media pipeline manager, a media pipeline controller and thelike. Meanwhile, various pipelines generated under the control of themedia pipeline manager and the media pipeline controller may include acamera preview pipeline, a cloud game pipeline, a media pipeline and thelike. Streaming protocol, auto/static gstreamer, DRM and the like may beincluded in the media pipeline, of which processing flow may bedetermined under the control of the path manager. The former descriptionwith reference to FIG. 10 is recited for a detailed processing processin the media server processor 1110/1210, which is not describedredundantly herein.

In the present specification, the resource manager in the media serverprocessor 1110/1210 can perform a resource managing with a counter basefor example.

FIG. 13 is a diagram illustrating a control method for a remote controldevice for controlling an arbitrary one among image display devicesaccording to embodiments of the present invention.

As shown in FIG. 13(a), a pointer 205 corresponding to the remotecontrol device 200 is displayed on the display unit 180.

A user may move or rotate the remote control device 200 vertically (asshown in FIG. 13(b)) or horizontally (as shown in FIG. 13(c)). Thepointer 205 displayed on the display unit 180 of the image displaydevice corresponds to a movement of the remote control device 200. Sincethe corresponding pointer 205 is moved and displayed according to amovement on a 3D space as show in the drawing, the remote control device200 may be referred to as a spatial remote controller.

FIG. 13(b) illustrate a case in which when a user moves the remotecontrol device 200 to the left, the pointer 205 displayed on the displayunit 180 of the image display device also moves to the left in responseto the user's movement.

Information on a movement of the remote control device 200 detectedthrough a sensor of the remote control device 200 is transmitted to theimage display device. The image display device may calculate thecoordinates of the pointer 205 from the information on the movement ofthe remote control device 200. The image display device may display thepointer 205 to match the calculated coordinates.

FIG. 13(c) illustrates a case in which while a specific button in theremote control device 200 is pressed, a user moves the remote controldevice 200 away from the display unit 180. By doing so, a selection areain the display unit 180 corresponding to the pointer 205 may be zoomedin and enlarged. On the other hand, when a user moves the remote controldevice 200 close to the display unit 180, a selection area in thedisplay unit 180 corresponding to the pointer 205 may be zoomed out andreduced. On the contrary, when the remote control device 200 is awayfrom the display unit 180, a selection area may be zoomed out, and whenthe remote control device 200 is close to the display unit 180, aselection area may be zoomed in.

Additionally, when a specific button in the remote control device 200 ispressed, the recognition of a vertical or horizontal movement may beexcluded. That is, when the remote control device 200 is moved away fromor close to the display unit 180, the up, down, left, or right movementmay not be recognized and only the back and forth movement may berecognized. While a specific button in the remote control device 200 isnot pressed, only the pointer 205 is moved according to the up, down,left or right movement of the remote control device 200.

Moreover, the moving speed or moving direction of the pointer 205 maycorrespond to the moving speed or moving direction of the remote controldevice 200.

Furthermore, the pointer 205 in this specification means an objectdisplayed on the display unit 180 in response to an operation of theremote control device 200. Accordingly, besides an arrow form displayedas the pointer 205 in the drawing, various forms of objects arepossible. For example, the above concept includes a point, a cursor, aprompt, and a thick outline. Then, the pointer 205 may be displayed onone point of a horizontal axis and a vertical axis on the display unit180 and also may be displayed on a plurality of points such as a lineand a surface.

FIG. 14 is a block diagram illustrating the inside of a remote controldevice for controlling an arbitrary one among image display devicesaccording to embodiments of the present invention.

As shown in FIG. 14, the remote control device 200 may include awireless communication unit 225, a user input unit 235, a sensor unit240, an output unit 250, a power supply unit 260, a storage unit 270,and a control unit 280.

The wireless communication unit 225 transmits and receives signals toand from any arbitrary one of the image display devices according to theaforementioned embodiments of the present invention. Hereinafter, adescription will be given by taking as an example an image displaydevice 100 among the image display devices according to the embodimentsof the present invention.

In the present embodiment, the remote control device 200 may include anRF module 221 configured to transmit and receive signals to and from theimage display device 100 according to the RF communication standards andan IR module 223 configured to transmit and receive signals to and fromthe image display device 100 according to IR the communicationstandards.

In addition, the remote control device 200 may transmit a signalcarrying information on the motions of the remote control device 200 tothe image display device 100 through the RF module 221.

Moreover, the remote control device 200 may receive a signal transmittedfrom the image display device 100 through the RF module 221.Additionally, if necessary, the remote control device 200 may transmitcommands for power on/off, channel change, volume change, and the liketo the image display device 100 through the IR module 223.

The user input unit 235 may be configured with a keypad, a button, atouch pad, or a touch screen. A user may manipulate the user input unit235 to input a command related to the image display device 100 throughthe remote control device 200. If the user input unit 235 includes ahard key button, the user may input a command related to the imagedisplay device 100 through the remote control device 200 using the pushoperation of the hard key button. If the user input unit 235 includes atouch screen, the user may input a command related to the image displaydevice 100 by touching a soft key of the touch screen through the remotecontrol device 200. Additionally, the user input unit 235 may alsoinclude diverse types of input means that can be manipulated by theuser, such as a scroll key or a jog key. Further, such examples given inthe description of the present invention will not limit the scope of thepresent invention.

The sensor unit 240 may include a gyro sensor 241 or an accelerationsensor 243.

The gyro sensor 241 may sense information on a movement of the remotecontrol device 200.

For example, the gyro sensor 241 may sense information on an operationof the remote control device 200 on the basis of x, y, and z axes andthe acceleration sensor 243 may sense information on a movement speed ofthe remote control device 200. Moreover, the remote control device 200may further include a distance measurement sensor to sense a distancefrom the display unit 180.

The output unit 250 may output image or voice signals corresponding tomanipulation of the user input unit 235 or signals transmitted from theimage display device 100. The user may recognize whether the user inputunit 235 is manipulated or the image display device 100 is controlledthrough the output unit 250.

For example, the output unit 250 may include an LED module 251 thatflashes, a vibration module 253 that generates vibration, a sound outputmodule 255 that outputs sound, or a display module 257 that outputs animage, if the user input unit 235 is manipulated or signals aretransmitted/received to/from the image display device 100 through thewireless communication unit 225.

The power supply unit 260 may supply power to the remote control device200 and if the remote control device 200 does not move during apredetermined time, stop the power supply, so that power consumption maybe reduced. The power supply unit 260 may resume power supply if apredetermined key disposed on the remote control device 200 ismanipulated.

The storage unit 270 may store various kinds of programs and applicationdata necessary for control or operation of the remote control device200. If the remote control device 200 transmits/receives signalswirelessly to/from the image display device 100 through the RF module221, the remote control device 200 and the image display device 100transmit and receive signals through a predetermined frequency band. Thecontrol unit 280 of the remote control device 200 may store, in thestorage unit 270, information on a frequency band fortransmitting/receiving signals to/from the image display device 100paired with the remote control device 200 and refer to it.

The control unit 280 may control overall operations for the remotecontrol device 200. The control unit 280 may transmit a signalcorresponding to a predetermined key manipulation of the user input unit235 or a signal corresponding to a movement of the remote control device200 sensed by the sensor unit 240 to the image display device 100through the wireless communication unit 225.

FIG. 15 is a block diagram illustrating the configuration of amultimedia device according to an embodiment of the present invention.

Referring to FIG. 15, a multimedia device 1500 according to anembodiment of the present invention includes a tuner 1510, acommunication module 1520, a controller 1530, a display module 1540, amemory 1550, and an EPG signal processing module 1560. It will of coursebe apparent that, if necessary, some of the modules shown in FIG. 15 maybe switched or deleted or other modules may be added, without departingfrom the scope of the appended claims and their equivalents. Moreover,for example, the multimedia device 1500 may be one of a television and aset top box (STB). Further, FIG. 15 could be supplementarily interpretedwith reference to the features described in FIG. 2.

The tuner 1510 receives a broadcast signal, an audio decoder (not shownin the drawing) decodes audio data included in the received broadcastsignal, and a video decoder (not shown in the drawing) performs decodesvideo data included in the received broadcast signal.

The display module 1540 displays the decoded video data on a first area,and an interface module (or the communication module 1520) receives atleast one command from an external device.

The controller 1530 controls at least one of the tuner 1510, the displaymodule 1540 and the interface module. In addition, the controller 1530executes a specific area enlargement mode in accordance with the atleast one command received from the external device. Moreover, thecontroller 1530 displays video data corresponding to the video data on asecond area within the first area. In this case, the second areaincludes an indicator, and the video data displayed on the first area ischanged depending on at least one of a location of the indicator and asize of the indicator.

According to another embodiment of the present invention, theabove-mentioned process can be applied to video data stored in thememory 1550 rather than the broadcast signal. In addition, thecontroller 1530 automatically executes the specific area enlargementmode in accordance with category information of the received broadcastsignal. The category information of the broadcast signal is designed tobe processed by the EPG signal processing module 1560.

For instance, the above-mentioned indicator is implemented as a graphicimage of a guide box guiding a specific area that will be or has beenenlarged. Details will be described later with reference to FIG. 19.

Depending on the video data of the received broadcast signal, thecontroller 1530 changes coordinate information of a pointer that movesin accordance with a motion of the external device. For instance, ifresolution information of the video data of the received broadcastsignal corresponds to high definition (HD), the coordinate informationof the pointer is designed to be scaled by 0.66 times. If the resolutioninformation of the video data of the received broadcast signalcorresponds to full high definition (FHD), the coordinate information ofthe pointer is designed to be scaled by 1 time. If the resolutioninformation of the video data of the received broadcast signalcorresponds to ultra-high definition (UHD), the coordinate informationof the pointer is designed to be scaled by 2 times. It will be describedin detail with reference to FIG. 27.

If an enlargement or reduction magnification of the video data displayedon the first area is changed in accordance with the at least one commandreceived from the external device after the specific area enlargementmode has been executed, the controller 1530 automatically changes thesize of the indicator in the second area. It will be described in detailwith reference to FIG. 31.

If the specific area to be enlarged is recognized within the first areain accordance with the at least one command received from the externaldevice after the specific area enlargement mode has been executed, thecontroller 1530 automatically changes the center point of the indicatorin the second area. It will be described in detail with reference toFIG. 32.

The controller 1530 controls both of the video data and the indicatorwithin the second area to be removed after the elapse of a predeterminedtime after executing the specific area enlargement mode or in accordancewith the at least one command received from the external device. It willbe described in detail with reference to FIG. 33.

Moreover, after both of the video data and the indicator within thesecond area have been removed, the controller 1530 displays a graphicimage for guiding that the specific area enlargement mode is beingexecuted. In this case, the graphic image includes informationindicating the enlargement magnification. The controller 1530 isdesigned to display both of the video data and the indicator on thesecond area again in accordance with a command for selecting the graphicimage. It will be described in detail with reference to FIG. 34.

For instance, the location or size of the indicator is changed based oninformation obtained from a touch sensor or motion sensor of theexternal device. For example, the external device can be designed asdescribed with reference to FIGS. 6, 13 and 14. In more particular, forexample, the external device corresponds to a remote controller or amobile device, which includes at least one of a radio frequency (RF)module and an infrared (IR) module.

For example, the aforementioned first area corresponds to a full screenof the television, and the second area corresponds to a partial areaincluded in the first area. Details will be further described withreference to FIG. 19.

FIG. 16 is a flowchart for explaining a control method for a multimediadevice according to an embodiment of the present invention. It will ofcourse be apparent that FIG. 16 could be supplementarily interpretedwith reference to the features described in FIG. 15.

Referring to FIG. 16, a multimedia device according to an embodiment ofthe present invention performs decoding on video data that is receivedexternally or stored in a memory [S1610], displays the decoded videodata on a first area [S1620], and then receives at least one commandfrom an external device [S1630]. For instance, the multimedia devicecorresponds to one of a television, an STB and the like.

Moreover, the multimedia device executes a specific area enlargementmode in accordance with the at least one commands received from theexternal device [S1640] and displays video data corresponding to thevideo data on a second area within the first area [S1650].

In addition, the second area includes an indicator, and the video datadisplayed on the first area is changed depending on at least one of alocation of the indicator and a size of the indicator. More details willbe further described with reference to FIG. 19.

FIG. 17 is a diagram illustrating a case in which a specific areaenlargement mode is activated according to an embodiment of the presentinvention.

As shown in the top left drawing 1710 of FIG. 17, in case the controller1530 receives a command for activating a specific area enlargement modefrom an external remote controller 1740 through the communication module1520, the controller 1530 displays at least one of a notificationmessage 1712 indicating that the specific area enlargement mode isactivated and a pointer 1714 capable of selecting a specific point to beenlarged on a first area 1710.

If the controller 1530 receives a command for specifying the specificpoint to be enlarged within the first area from the external remotecontroller 1740 through the communication module 1520, the controller1530 specifies an area including the specific point using the pointer inresponse to the command, enlarges the specified area including thespecific point, and then displays the enlarged specified area includingthe specific point.

As shown in the top right drawing 1720 of FIG. 17, in case thecontroller 1530 receives a command for deactivating the specific areaenlargement mode from the external remote controller 1740 through thecommunication module 1520, the controller 1530 displays a notificationmessage 1722 indicating that the specific area enlargement mode isdeactivated. A pointer 1724 capable of selecting a specific point isdesigned to be removed within a first area 1720.

As shown in the bottom left drawing 1730 of FIG. 17, the controller 1530receives a command for selecting a specific part of the first area usinga pointer 1734 from the external remote controller 1740 through thecommunication module 1520 and then displays a specific area 1732 to beenlarged in response to the received command in advance. Therefore, fromthe perspective of a user, it is advantageous in that the user can checkan area to be enlarged in advance.

FIG. 18 is a diagram illustrating a case in which a pointer shape ischanged when a specific area enlargement mode is activated according toan embodiment of the present invention.

As shown in FIG. 18, if a specific area enlargement mode 1840 isactivated by a command received from an external remote controller 1830,the controller 1530 changes a pointer shape from an original shape(first graphic image) to a different shape (second graphic image).

For instance, if a magnification ratio of the specific area enlargementmode increases, the controller 1530 changes a pointer shape from anoriginal shape 1810 to the ‘+’ shape 1820.

If the magnification ratio of the specific area enlargement modedecreases, the controller 1530 changes the pointer shape from theoriginal mode to the ‘−’ shape.

That is, according to an embodiment of the present invention, if thespecific area enlargement mode is activated, the pointer shape ischanged to a magnifying glass shape. In addition, the pointer shapevaries depending on an increase and decrease in the magnification ratio.Therefore, the user can intuitively know whether the specific areaenlargement mode is initiated and whether the magnification ratioincreases, thereby improving user convenience.

FIG. 19 is a diagram illustrating an example of controlling a screenwhen a specific area enlargement mode is activated according to anembodiment of the present invention. Hereinafter, ‘specific areaenlargement mode’ can be referred to as ‘enlargement mode’ forconvenience of description.

First of all, a display device according to an embodiment of the presentinvention displays a content on a main screen 1910 and enters anenlargement mode in accordance with an enlargement input requestreceived from a remote controller.

The display device displays a window 1920 including the contentdisplayed on the main screen 1910 and an indicator 1930 for selecting aspecific area of the displayed content within the displayed window 1920.

The display device enlarges the selected specific area of the displayedcontent. In addition, the display device is designed to display theenlarged selected specific area of the displayed content on the mainscreen 1910. The above-mentioned content corresponds to a video.

For convenience of description, the main screen 1910 can be referred toas a first region and the window 1920 can be referred to as a secondregion. There are no limitations on the form and size of the window1920.

In particular, for example, if the display device receives an inputsignal (e.g., ‘OK’ button) from a remote controller 1940 during apredetermined time or more while displaying video data included in abroadcast signal on the main screen 1910, the display device displaysthe video data even on the window 1920. When the specific areaenlargement mode is initially executed, the same video data is displayedon both of the main screen 1910 and the window 1920. In particular, thevideo data displayed on the main screen 1910 is identical to the thatdisplayed on the window 1920, but the video data has a different size.

Moreover, an indicator 1930 is displayed on the window 1920, and theindicator 1930 is used by a user to select a specific area that the userdesires to enlarge. For instance, the indicator 1930 can be implementedas a graphic image of a guide box guiding a specific area that will beor has been enlarged. Moreover, it is apparent that the adoption ofother graphic images comes within the scope of the appended claims andtheir equivalents.

Furthermore, the window 1920 may be named a total window, and theindicator 1930 may be named a local window. A specified area is enlargedthrough the indicator 1930 and then displayed on the main screen 1910.In other words, when the display device newly enters the specific areaenlargement mode, the display device outputs original video data throughthe main screen 1910. When the specific area to be enlarged is selectedthrough the window 1920 and the indicator 1930, the display devicereplaces the original video data with the video data where only thespecific area is enlarged. That is, the display device displays thevideo data instead of the original video data. Further, for example, thedisplay device is designed to change a location of the indicator 1930when a location of a pointer 1932 is changed. Additionally, if thespecific area to be enlarged is decided after executing the specificarea enlargement mode, the original video data is displayed on thewindow 1920 by being scaled by 1 time. This means that only the size ofthe original video data is reduced. Thereafter, the video data where thespecific area is enlarged (by greater than 1 time, e.g., 1.2, 1.5 or 2.0times) is displayed on the main screen 1910 instead of the originalvideo data.

For instance, the controller 1530 is designed to allow the user toeasily check the specific area that the user desires to enlarge byadjusting brightness of the inside of the indicator 1930 in the window1920 to be high and brightness of the outside of the indicator 1930 tobe low.

Moreover, a ratio indicator bar including both an enlargement button1924 and a reduction button 1922 for changing a screen magnificationratio is present on the main screen 1910. If the controller 1530receives a command for selecting one of the reduction button 1922 andthe enlargement button 1924 using the pointer 1932, the controller 1530adjusts a size of the indicator 1930 in response to the commandaccording to a predetermined ratio. The controller 1530 changes a shapeof the pointer 1932 according to the command for selecting either thereduction button 1922 and the enlargement button 1924. The ratioindicator/control bar including both of the reduction button 1922 andthe enlargement button 1924 is named a second indicator, and it could bedistinguished from indicator 1930 mentioned in the foregoingdescription.

In this case, a minimum value of magnification in accordance with theselection of the reduction button 1922 can be 1 time and a maximum valueof magnification in accordance with the selection of the enlargementbutton 1924 can be 5 times. In this case, the maximum value of theenlargement magnification is not limited to 5 times but can beadjustable. In case the enlargement magnification is smaller than 1time, an image displayed on the screen is reduced.

For instance, if the controller 1530 receives a command for selectingthe reduction button 1922, the controller changes the pointer shape froman original shape of the pointer 1932 to the ‘−’ shape. If thecontroller 1530 receives a command for selecting the enlargement button1924, the controller 1530 changes the pointer shape from the originalshape of the pointer 1932 to the ‘+’ shape.

Moreover, the window 1920 can be implemented as a PIP screen, and a sizeof the PIP screen is designed to be adjustable. For instance, if thecontroller 1530 receives a command for clicking on an edge portion(i.e., corner portion) of the PIP screen and moving it from a firstpoint to a second point different from the first point in a state inwhich clicking is maintained from an external device (e.g., remotecontroller), the controller 1530 is able to adjust the size of the PIPscreen.

Furthermore, the controller 1530 can change a location of the PIPscreen.

For instance, if the controller 1530 receives a command for clicking onthe first point of the PIP screen by the pointer and moving it from thefirst point to the second point within a first area different from thefirst point in a state in which clicking is maintained from the externalremote controller, the controller 1530 can change the location of thePIP screen. In this case, the above-mentioned PIP screen corresponds tothe window 1920 shown in FIG. 19.

For example, if the window 1920 is continuously present, it may causeinconvenience to the user in watching the played vide data. Therefore,after elapse of a predetermined time (e.g., three seconds), thecontroller 1530 switch states of the widow 1920 and the indicator 1930into a hidden state. Thereafter, if the controller 1530 receives apredetermined command from an external device 1940, the controller 1530displays the window 1920 and the indicator 1930 on the PIP screen again.

If the pointer 1932 is located at one of a right boundary line, a leftboundary line, an upper boundary line, and a lower boundary line of thesecond area 1920, the controller 1530 changes the states of the widow1920 and the indicator 1930 into the hidden state. If a specific commandis received from the external device 1940, the controller 1530 displaysthe window 1920 as the PIP screen in the main screen 1910 again.

The controller 1530 moves the indicator 1930 using the pointer 1932, andwhen the location of the indicator 1930 is changed, the video datadisplayed on the main screen 1910 is also changed. For example, thevideo data in the area specified by the indicator 1930 and the enlargedvideo data displayed on the main screen 1910 are identical to each otherbut has different sizes (when the main screen 1910 and the indicator1930 shown in FIG. 19 are compared, it could be more apparent). Inparticular, for example, if the indicator 1930 in the window 1920includes a specific object only, the main screen 1910 also displaysvideo data including the specific object only. However, when it iscompared with video data within the indicator 1930, the video datawithin the main screen 1910 is enlarged only.

Therefore, it is possible to obtain an advantage of checking theenlarged specific area of the original video data more rapidly bydisplaying the changed location and size of the indicator 1930 in realtime.

In other words, if the specific area enlargement mode is executed, theoriginal data is displayed on both of the main screen 1910 and thewindow 1920. However, the video data with the reduced size is displayedon the window 1920. In order to enlarge the specific area, the pointer1932 can be located within the main screen 1910 or the window 1920. Thespecific area to be enlarged is confirmed by the pointer 1932 as thecenter point.

If the specific area to be enlarged is confirmed, the video data wherethe specific area is enlarged is displayed on the main screen 1910instead of the original video data. Moreover, the display device can bedesigned such that the enlarged video data displayed on the main screen1910 is replaced again with the original video data by adjusting themagnification ratio. When the original video data is displayed on themain screen 1910 again, the user can newly designate a specific area tobe enlarged by selecting a random point in the main screen 1910. It is amatter of course that designation of the specific area to be enlargedusing the indicator 1930 in the window 1920 comes within the scope ofthe appended claims and their equivalents.

Furthermore, if an enlargement/reduction magnification is adjusted usingthe external device 1940 while the video data where the specific area isenlarged is displayed on the main screen 1910, the display deviceautomatically changes the size of the indicator 1930 in the window 1920.Therefore, the user has an advantage of easily checking which portion ofthe window 1920 corresponds to video data enlarged or reduced on themain screen 1910 in real time.

The second indicator 1922/1924 shown in FIG. 19 is used for setting amagnification level, and a content to be displayed on the main screen1910 is received through a tuner or an external device. For instance,the external device corresponds to at least one of an STB, a PC and acellular phone.

The size of the indicator 1930 is automatically changed depending on themagnification level selected through the second indicator 1922/1924.

Furthermore, although not shown in FIG. 16, another embodiment includingthe steps of receiving a first magnification level for enlarging thedisplayed content, displaying an magnification indicator having a firstdisplay size based on the received first magnification level, receivinga second magnification level for enlarging the displayed content, anddisplaying an magnification indicator having a second display sizedifferent from the first display size based on the received secondmagnification level can be implemented with reference to FIG. 19.

For instance, the window 1920 includes a picture in picture (PIP)window.

The movement of the window 1920 within the main screen 1910 also comeswithin the scope of the appended claims and their equivalents. Inaddition, the movement of the indicator 1930 within the window 1920,which is performed to select another specific area of a contentdisplayed on the window 1920, comes within the scope of the appendedclaims and their equivalents as well.

The indicator 1930 moves in accordance with a pointer signal receivedfrom the remote controller 1940, and the size of the indicator 1930 ischanged according to a wheel signal received from the remote controller1940.

The size of the indicator 1930 is increased in accordance with a reducedmagnification level 1922. On the contrary, the size of the indicator1930 is reduced in accordance with an increased magnification level1924.

For instance, the indicator 1930 is implemented as a graphic image of aguide box guiding a specific area that will be or has been enlarged.

In another aspect of the present invention, the step of changingcoordinate information of the pointer that moves in accordance with amotion of the remote controller according to the video data of thecontent displayed on the main screen 1910 shown in FIG. 19 is furtherincluded. For instance, if resolution information of the video data ofthe content corresponds to HD, the step of scaling the coordinateinformation of the pointer by 0.66 times is further included. If theresolution information of the video data of the content corresponds toFHD, the step of scaling the coordinate information of the pointer by 1time is further included. If the resolution information of the videodata of the content corresponds to UHD, the step of scaling thecoordinate information of the pointer by 2 times is further included.More details will be further described with reference to FIG. 27.

Both of the window 1920 and the indicator 1930 are controlled to beremoved after the elapse of a predetermined time after execution of theenlargement mode or in accordance with at least one command receivedfrom the remote controller 1940. After both of the window 1920 and theindicator 1930 has been removed, a graphic image for guiding that theenlargement mode is being executed is displayed. In addition, thegraphic image includes information indicating an enlargementmagnification. Both of the window 1920 and the indicator 1930 aredisplayed again according to a command for selecting the graphic image.More details will be further described with reference to FIGS. 31 to 34.

FIG. 20 is a diagram illustrating an example of moving a specific pointon an enlarged screen using a pointer when a specific area enlargementmode is activated according to an embodiment of the present invention.

Referring to FIG. 20, if the controller 1530 receives a command forselecting a specific point 2012 of a full screen using a pointer from anexternal device while displaying an area specified by an indicator 2030on a first area 2010 through the full screen, the controller 1530 movesa center point of the area specified by the indicator 2030 from anexisting center point to the specific point 2012. Subsequently, thecontroller 1530 generates a new enlargement area with respect to thespecific area 2012 and displays the generated new enlargement areathrough the full screen.

Further, according to another embodiment of the present invention, acenter point of a specific area to be enlarged can be selected in asecond area 2020, or the center point of the specific area to beenlarged can be selected in the first area 2010. When the center pointof the specific area to be enlarged is selected using the first area2010, it has an advantage of adjusting the enlargement area minutely. Onthe other hand, when the center point of the specific area to beenlarged is selected using the second area 2020, it has an advantage ofchanging the specific area while checking original video data.

FIG. 21 is a diagram illustrating an example of controlling a screenusing a remote controller when a specific area enlargement mode isactivated according to an embodiment of the present invention. Asdescribed above, a multimedia device (e.g., TV or STB) according to anembodiment of the present invention is controlled by an external device,and the external device corresponds to a remote controller or a mobiledevice. Although FIG. 21 shows a remote controller as an example of anexternal device, the scope of the appended claims and their equivalentsis not limited to the remote controller.

According to an embodiment of the present invention, an external remotecontroller 2140 includes a wheel key 2142, a direction key 2144 and avolume key 2146.

If the controller 1530 receives a specific command corresponding tomanipulation of the wheel key 2142 from the external remote controller2140, the controller 1530 adjusts a screen magnification ratio inaccordance with the manipulation of the wheel key 2142.

For example, if the controller 1530 receives a specific commandcorresponding to an input for rotating a wheel in the upper direction ofthe wheel key 2142 from the external remote controller 2140, thecontroller 1530 increases the screen magnification ratio. If thecontroller 1530 receives a specific command corresponding to an inputfor rotating the wheel in the lower direction of the wheel key 2142 fromthe external remote controller 2140, the controller 1530 decreases thescreen magnification ratio.

The user can change the screen magnification ratio in the range of 1time to 5 times through the wheel key of the remote controller. Wheneverthe wheel key is moved by 1 unit, the screen magnification ratio ischanged by 0.2 times. The screen magnification ratio is not fixed but isdesigned to be capable of being modified through a user configuration.

If the controller 1530 receives a specific command corresponding tomanipulation of the volume key 2146 from the external remote controller2140, the controller 1530 adjusts the screen magnification ratio inaccordance with the manipulation of the volume key 2146.

For example, if the controller 1530 receives a specific commandcorresponding to the ‘+’ part of the volume key 2146 from the externalremote controller 2140, the controller 1530 increases the screenmagnification ratio. On the other hand, if the controller 1530 receivesa specific command corresponding to the ‘−’ part of the volume key 2146from the external remote controller 2140, the controller 1530 decreasesthe screen magnification ratio.

If the controller 1530 receives a specific command corresponding tomanipulation of the direction key 2144 from the external remotecontroller 2140, the controller 1530 moves a center point of an areaspecified by an indicator 2130 from an existing center point to aspecific point in accordance with the manipulation of the direction key2144, generates enlarged video data with respect to the specific point,and then displays the generated enlarged video data on a first area2110.

If the magnification ratio and location of the specific area are changedusing keys of the external remote controller, the location and size ofthe indicator 2130 within a second area 2120 corresponding to the PIPscreen are also changed. According to another embodiment of the presentinvention, an external remote controller 2150 includes a volume key2156, a channel key 2152 and a touch pad 2154. It is a matter of coursethat the external remote controller 2150 is controlled by a motionsensor or a voice recognition sensor.

If the controller 1530 receives a specific command corresponding tomanipulation of the volume key 2156 from the external remote controller2150, the controller 1530 adjusts the screen magnification ratio inaccordance with the manipulation of the volume key 2156.

For example, if the controller 1530 receives a specific commandcorresponding to a part at the top of the volume key 2156 from theexternal remote controller 2150, the controller 1530 increases thescreen magnification ratio. If the controller 1530 receives a specificcommand corresponding to a part at the bottom of the volume key 2156from the external remote controller 2150, the controller 1530 decreasesthe screen magnification ratio.

If the controller 1530 receives a specific command corresponding tomanipulation of the channel key 2152, the controller 1530 adjusts thescreen magnification ratio in accordance with the manipulation of thechannel key 2152.

For example, if the controller 1530 receives a specific commandcorresponding to a part at the top of the channel key 2152 from theexternal remote controller 2150, the controller 1530 increases thescreen magnification ratio. If the controller 1530 receives a specificcommand corresponding to a part at the bottom of the channel key 2152from the external remote controller 2150, the controller 1530 decreasesthe screen magnification ratio.

If the controller 1530 receives a specific command corresponding tomanipulation of the touch pad 2154 from the external remote controller2140, the controller 1530 moves a center point of an area specified bythe indicator 2130 from an existing center point to a specific point inaccordance with the operation of the touch pad 2154, generates enlargedvideo data centering on the specific point, and then displays thegenerated enlarged video data on the first area 2110.

FIG. 22 is a diagram illustrating an example of automatically executinga specific area enlargement mode through association with EPGinformation according to an embodiment of the present invention.

Referring to FIG. 22, the EPG signal processing module 1560 extractscategory information (e.g., genre information, etc.) from a broadcastsignal containing an EPG signal and then analyzes the extractedcategory. In this case, for example, categories include sports, news,documentary, movie, drama, entertainment, art, talk show and the like.

If the information included in the broadcast signal corresponds to aspecific category, the controller 1530 executes the specific areaenlargement mode automatically.

For example, if the currently displayed broadcast program (i.e., videodata) corresponds to such a category as sports, news or the like, thecontroller 1530 activates the specific area enlargement modeautomatically.

Moreover, if the currently displayed broadcast program (i.e., videodata) corresponds to one of adult video, violent video, adult action andX-rated content, the controller 1530 switches a state of the specificarea enlargement mode to an off state.

Therefore, according to an embodiment of the present invention, it hasan advantage of minimizing a time required for entering the specificarea enlargement mode or reducing the misuse of the present invention bydesigning that the specific area enlargement mode is automaticallyturned on or off in accordance with the category information (e.g.,genre information) of video data. FIG. 23 is a diagram illustrating anexample of executing a specific area enlargement mode throughassociation with a time shift function according to an embodiment of thepresent invention.

In this case, a time shift function means the function of enabling auser to watch a missing program in the course of watching TV in realtime. For example, the memory 1550 is designed to automatically store acurrently displayed broadcast program for a prescribed time even thoughthe memory 1550 does not receive an explicit save command from the user.In this case, for example, the memory 1550 includes a removable harddisk, an external USB memory, a memory built in a multimedia device, andthe like.

The controller 1530 displays a bar 2310 indicating a playback time onthe bottom of a first area 2300, on which video data is displayed. Forexample, in case genre information of the video data corresponds tosports, the controller 1530 marks a time 2312 at which a goal is scoredand a time 2314 at which the corresponding video data is watched mostpreferably. The specific times 2312 and 2314 can be collected throughEPG information or a web search. The specific area enlargement mode isdesigned to be automatically executed when the specific times areselected.

According to another embodiment of the present invention, the controller1530 retrieves at least one piece of video data stored in the memory1550 and plays a part of the retrieved video data where the specificarea enlargement function is performed.

For example, in the case of video data for a specific singer groupconsisting of 9 members including first and second singers, the user maybe interested in only a part where the first and second singers sing.Unlike the conventional time shift function, the controller 1530 storesinformation on the part where the specific area enlargement function isperformed together with the video data in the memory 1550.

The controller 1530 retrieves the part where the specific areaenlargement function is performed from the memory 1550 and plays theretrieved part only.

Therefore, according to the present invention, since the part where thespecific area enlargement function is automatically performed isretrieved, and the retrieved section is played only, it has anadvantageous in that the user does not need to play all part of thevideo data.

According to a further embodiment of the present invention, thecontroller 1530 partitions the full screen based on the number ofspecific area enlargement functions executed for video data temporarilystored in the memory. In addition, the controller 1530 is designed suchthat the video data where the specific area enlargement function isperformed (i.e., the video data where only the specific area isenlarged) is displayed through each of the partitioned screens. Forexample, regarding one piece of video data (e.g., broadcast program), ifa total number of times of execution of the specific area enlargementfunction is 9, the controller 1530 displays 9 partitioned screens.Thereafter, the controller 1530 displays each part where the specificarea enlargement function is performed through each of the 9 partitionedscreens.

Hence, it has an advantage of checking a part where the specific areaenlargement function is performed more rapidly.

FIG. 24 is a diagram illustrating an example of switching between a fullscreen and a zoom screen according to an embodiment of the presentinvention.

As shown in FIG. 24, if the controller 1530 receives a specific commandfrom an external remote controller, the controller 1530 switches a videosignal to be transmitted to a first area and a video signal to betransmitted to a second area from each other and displays each of thefirst area and the second area.

In particular, the video signal to be transmitted to the first areacorresponds to video data where a specific area is enlarged, and thevideo signal to be transmitted to the second area corresponds to areduced size of the original video data.

Thus, as shown in the top drawing 2410 of FIG. 24, the video data havingthe enlarged specific area is displayed on the main screen and thereduced size of the original video data is displayed on the PIP screen.Specifically, a full screen that is reduced by a prescribed ratio and alocation of the enlarged area of the full screen are displayed on thePIP screen.

As shown in the bottom drawing 2420 of FIG. 24, the full screen isdisplayed on the main screen and a screen with the enlarged specificarea is displayed on the PIP screen.

Therefore, according to an embodiment of the present invention, it hasan advantage in that if necessary, the original video data or the videodata with the enlarged specific area is selectively displayed on thefull screen or the PIP screen by rapidly switching therebetween.

FIG. 25 is a diagram illustrating an example of selecting a plurality ofpoints on a screen and enlarging the selected points according to anembodiment of the present invention.

Referring to FIG. 25, if the controller 1530 receives a command forspecifying a plurality of points in a first area 2510 from an externalremote controller through a communication module within a predeterminedtime in a state in which a specific area enlargement mode is activated,the controller 1530 automatically generates and displays PIP screensamounting to the number of the points.

For instance, if the controller 1530 receives a command for selectingthree specific points in the first area 2510 from the external remotecontroller through the communication module within three seconds afterentering the specific area enlargement mode, the controller 1530displays a first PIP screen 2520, a second PIP screen 2530, and a thirdPIP screen 2540. Each of the PIP screens includes video data enlargedwith respect to each of the three specific points.

According to an embodiment of the present invention, if a user desiresto watch a screen by enlarging a plurality of points within the screen,the user can specify the plurality of points. Thereafter, each of thespecified points can be displayed on a specific part of the screen as aPIP screen.

In this case, if multiple people are located at different positions ofthe screen, the user can simultaneously specify the multiple people andidentify the specified multiple people. By doing so, the user can knowdetailed information on clothing, watches and accessories of theidentified people more precisely, thereby improving user convenience.

FIG. 26 is a diagram illustrating an example of selecting a plurality ofpoints on a screen and enlarging the selected points according to anembodiment of the present invention. Since the embodiment of FIG. 26 issimilar with that of FIG. 25, a description will be given a differencebetween the two embodiments only. However, FIG. 26 could besupplementarily interpreted with reference to the features described inFIG. 25.

For example, if the controller 1530 receives a command for selectingthree specific points from an external remote controller through acommunication module within three seconds after entering the specificarea enlargement mode, the controller 1530 reduces a size of a firstarea 2610 where original video data is displayed by 80% and thendisplays a first subscreen 2620, a second subscreen 2630 and a thirdsubscreen 2640 on an area except the first area 2610. Each of the PIPscreens includes video data enlarged with respect to each of the threespecific points.

Compared with FIG. 25, a solution for solving the problem that theoriginal vide data is covered with the PIP screens is provided in FIG.26. In particular, the size change of the first area where the originalvide data is displayed in accordance with the number of subscreens(e.g., subscreens 2620, 2630 and 2640) could be interpreted as onefeature of the present invention.

FIG. 27 is a diagram illustrating a solution for a case in whichcoordinates from a remote controller do not match those of an inputvideo according to an embodiment of the present invention. In a processfor implementing another embodiment of the present invention, technicalproblems that will be mentioned with reference to FIG. 27 and in thefollowing description should be solved.

Referring to FIG. 27, coordinates of a remote controller are 1920×1080in a two-dimensional plane 2710, whereas coordinates of a video signalmay become 3840×2160 in a two-dimensional plane 2720. In this case, thecoordinates are not fixed but can be modified in accordance with aresolution of an input video signal or a device. Each of values of thecoordinates is not an absolute value but can be changed as a relativevalue. The resolution means how many pixels are included in a screen andis represented by multiplying the number of width pixels and the numberof height pixels. That is, if a resolution is 1920×1080, the number ofwidth pixels is 1920, and the number of height pixels is 1080. Inaddition, it is represented as two-dimensional plane coordinates.

For example, since the coordinates of the remote controller are notequal to those of the video signal, even if a user selects a point P ofx=1440 and y=270, the controller 1530 recognizes that a point P′ ofx=720 and y=135 is selected.

Therefore, there may be a difference between the coordinates intended bythe user and the coordinates recognized by the controller 1530.

In this case, when the external remote controller intends to transferdata to the display device, the external remote controller transfers thedata by including coordinate information of the corresponding remotecontroller in the data. The external remote controller and the displaydevice are connected with each other through wireless communication, andthe wireless communication includes RF communication and IRcommunication. Moreover, the external remote controller may correspondto a mobile device such as a smartphone, a tablet PC, etc.

The controller 1530 performs scaling on the coordinate information ofthe external remote controller according to video signal information ofa content.

Specifically, when the video signal information of the content ischanged, the controller 1530 senses the changed video signal informationand then performs scaling on received plane coordinates of the externalremote controller based on the sensed video signal information.

For example, when the coordinates of the remote controller are 1920×1080and the video signal resolution information of the content is 720P HDcorresponding to 1280×720, the controller 1530 changes the receivedcoordinates of the remote controller to 1280×720 by scaling the receivedcoordinates of the remote controller with reference to the video signalinformation. When the resolution corresponds to HD, a scaling factorbecomes 0.66.

When the video signal resolution information of the content is FHDcorresponding to 1920×1080, the controller 1530 performs scaling withreference to the video signal information. When the resolution is FHD,the coordinates of the remote controller and the coordinates of thevideo signal information are the same, and thus, the scaling factorbecomes 1.

If the video signal resolution information of the content is UHDcorresponding to 3840×2160, the controller 1530 changes the receivedcoordinates of the remote controller to 3840×2160 by scaling thereceived coordinates of the remote controller with reference to thevideo signal information. When the resolution corresponds to UHD, thescaling factor becomes 2.

FIG. 28 is a diagram illustrating a solution for a case in which aspecific area to be enlarged is out of a video output range according toan embodiment of the present invention.

Referring to the top drawing 2810 of FIG. 28, when a specific area isenlarged with respect to a point at which a pointer is located, i.e., byconsidering the point as the center point, there may be an area which isnot included in original vide data.

Thus, as shown in the bottom drawing 2820 of FIG. 28, the specific areais enlarged by moving the center point to another point 2824 instead ofthe point 2822 at which the pointer is located. Compared with the topdrawing 2810 of FIG. 28, it is advantageous in that only an areaincluded in the original video data is enlarged.

FIG. 29 is a diagram illustrating an example of dividing a screen into aprescribed number of partial screens, enlarging a selected screen if auser selects the screen from the divided screens, and displaying theenlarged screen when video data is outputted according to an embodimentof the present invention.

Referring to the top drawing 2910 of FIG. 29, if the controller 1530receives a specific command from an external remote controller, thecontroller 1530 partitions a screen into nine screens to display videodata. If a user selects a specific screen 2912 from the partitionedscreens and the controller 1530 receives a command for selecting thespecific screen 2912 from the external remote controller, the controller1530 enlarges video data corresponding to the selected specific screen2912 and then displays the enlarged video data.

Moreover, as shown in the bottom drawing of FIG. 29, the controller 1530reduces original video data at a prescribed ratio and then displays thereduced original video data on a second area 2924. In addition, thecontroller 1530 enlarges the video data of the selected specific area asshown in the upper drawing 2910 of FIG. 29 and then displays theenlarged video data on a first area 2920. Moreover, as mentioned in theforegoing description, an indicator 2922 for guiding the enlargedspecific area is displayed together on the second area 2924.

FIG. 30 is a diagram illustrating an example in which a controllerdivides a screen into four, nine, or sixteen partial screens accordingto selection from a user and controls the divided screens when videodata is outputted according to an embodiment of the present invention.

As shown in FIG. 30, if the controller 1530 receives a specific command3020 from an external remote controller 3010, the controller partitionsa screen into four screens 3030 and then displays the four screens tooutput video data. Alternatively, the controller 1530 partitions thescreen into nine screens 3040 and then displays the nine screens.Further, the controller 1530 partitions the screen into sixteen screens3050 and then displays the sixteen screens. The number of partitionedscreens can be determined by a predetermined value or designed to beuser-selectable. With reference to the partitioned screens, a user canselect a specific area of the video data that the user desires toenlarge.

If an enlargement or reduction magnification of the video data displayedon a first area is changed in accordance with at least one commandreceived from an external device after a specific area enlargement modeis executed, the controller 1530 automatically changes a size of anindicator in a second area. It will be described in detail withreference to FIG. 31.

If the specific area to be enlarged is recognized within the first areain accordance with the at least one command received from the externaldevice after the specific area enlargement mode is executed, thecontroller 1530 automatically changes a center point of the indicator inthe second area. It will be described in detail with reference to FIG.32.

The controller 1530 controls both of the video data and the indicatorwithin the second area to be removed after the elapse of a predeterminedtime after execution of the specific area enlargement mode or inaccordance with the at least one command received from the externaldevice. It will be described in detail with reference to FIG. 33.

Moreover, after both of the video data and the indicator within thesecond area is removed, the controller 1530 displays a graphic image forguiding that the specific area enlargement mode is running. In thiscase, the graphic image includes information on enlargementmagnification. The controller 1530 is designed to display both of thevideo data and the indicator within the second area again in accordancewith a command for selecting the graphic image. It will be described indetail with reference to FIG. 34.

FIG. 31 is a diagram illustrating a process for adjusting amagnification ratio while a specific area enlargement mode is executedaccording to an embodiment of the present invention. It is apparent thatan embodiment partially different from the embodiment shown in FIG. 31,which is implemented by those skilled in the art with reference to theabove drawings, comes within the scope of the appended claims and theirequivalents.

As shown FIG. 31, when a specific area enlargement mode is executed anda specific area to be enlarged is also specified, video data of an areaspecified by an indicator 3130 within a second area 3120 is displayed ona first area 3110 as well. As mentioned in the foregoing description,the video data in the indicator 3130 matches, i.e., is identical to thevideo data in the first area 3110, but the two pieces of vide data has adifferent size.

Moreover, for the video data displayed on the first area 3110, anenlargement or reduction command can be additionally transmitted to amultimedia device (e.g., TV or STB) using an external device 3100. Forexample, a command for enlarging the video data in the first area 3110is generated through a specific button of the external device 3100 asshown in FIG. 31, and then, the corresponding command is transmitted tothe multimedia device.

Thus, as shown in FIG. 31, compared to the previous video data in thefirst area 3110, enlarged video data is displayed on the first area3111. Moreover, according to a feature of the present invention, atleast one of a size and location of the indicator 3131 within the secondarea 3121 is automatically changed depending on the video data withinthe first area 3111.

FIG. 32 is a diagram illustrating a process for selecting an area to beenlarged while a specific area enlargement mode is executed according toan embodiment of the present invention. It is apparent that anembodiment partially different from the embodiment shown in FIG. 32,which is implemented by those skilled in the art with reference to theabove drawings, comes within the scope of the appended claims and theirequivalents.

As described above or shown in FIG. 32, video data in a first area 3210and video data in a second area 3220 are identical to each other but thetwo pieces of video data has a different when a specific areaenlargement mode is initially executed. In addition, a location of anindicator 3230 in the second area 3220 is designed to correspond to aspecific area last selected in previously executed specific areaenlargement mode or a random area.

Meanwhile, it is assumed that a specific area to be enlarged is selectedusing an external device 3200 in the above-described situation.

Therefore, as shown in FIG. 32, video data where the specific area isenlarged is displayed on a first area 3211 and original video data isconsistently displayed on a second area 3221 like a previous second area3220. However, at least one of a location and size of an indicator 3231with the second area 3221 is automatically changed. In addition, asshown in FIG. 32, the location or size of the indicator 3231 isautomatically changed according to the video data displayed on the firstarea 3211. The above-described design has an advantage of rapidly andeasily checking which specific area of the original video data isenlarged and watched.

FIG. 33 is a diagram illustrating a process for eliminating a relatedindicator while a specific area enlargement mode is executed accordingto an embodiment of the present invention. It is apparent that anembodiment partially different from the embodiment shown in FIG. 33,which is implemented by those skilled in the art with reference to theabove drawings, comes within the scope of the appended claims and theirequivalents.

As described above or shown in FIG. 33, after a specific areaenlargement mode is executed, original video data is displayed on asecond area 3330, and video data where only a specific area is enlargedis displayed on a first area 3310. Moreover, a size and location of anindicator 3320 are changed according to the first area 3310.

However, there may be a problem that a part of the enlarged video dataoverlaps with the video data and the indicator 3320 within the secondarea 3330. To solve this problem, the multimedia device can be designedsuch that when a predetermined time (e.g., 3 to 5 seconds) elapses orwhen the multimedia device receives a specific command from an externaldevice 3300, the video data where the specific area is enlarged is stilldisplayed on the first area 3311, and the indicator and the originalvideo data are not displayed on a second area 3321 unlike to theprevious second area 3320. Therefore, it is expected that the effect ofenabling a user to watch the video data where the specific area isenlarged through only the first area 3311 can be achieved.

FIG. 34 is a diagram illustrating a process for displaying an eliminatedrelated indicator again while a specific area enlargement mode isexecuted according to an embodiment of the present invention. It isapparent that an embodiment partially different from the embodimentshown in FIG. 34, which is implemented by those skilled in the art withreference to the above drawings, comes within the scope of the appendedclaims and their equivalents. In particular, in FIG. 34, the casedescribed in FIG. 33 is assumed.

Compared to the bottom drawing of FIG. 33, FIG. 34 is equal to FIG. 33in that video data where a specific area is enlarged is displayed on afirst area 3410. However, FIG. 34 is different from FIG. 33 in thatthere is a second indicator 3440 for guiding that a specific areaenlargement mode is running. In particular, the second indicator 3440includes information indicating how many times the video data displayedon the first area 3410 is enlarged in comparison with original videodata.

In this case, if the second indicator 3440 is selected using an externaldevice 3400, the original video data is displayed on a second area 3430again, and an indicator 3420 corresponding to a first area 3411 is alsodisplayed again.

FIG. 35 is a block diagram illustrating the configuration of a displaydevice according to an embodiment of the present invention.

As shown in FIG. 35, a display device 3500 includes an interface module3510, a controller 3520, a display module 3530, and a memory 3540.

The interface module 3510 receives an input for requesting enlargementand an input for selecting a specific object from among at least oneobject displayed on the main screen from the remote controller.

The display module 3530 displays a content including at least one objecton the main screen of the display device.

The memory 3540 stores a content that has been broadcasted or a contentthat is currently broadcasted. In addition, the memory 3540 storesfeatures of the specific object according to a control command from thecontroller 3520.

The controller 3520 controls the display module 3530 and the interfacemodule 3510.

In addition, the controller 3520 enters an enlargement mode according tothe input for requesting the enlargement received from the remotecontroller, specifies the specific object in response to the receivedselection input, and enlarges a specific area including the specificobject.

Moreover, the controller 3520 detects the specific object in response tothe received selection input, specifies the detected specific object,and enlarges the specific area including the specific object.

In this case, the specific area may be changed according to a locationof the specific object.

Further, the controller 3520 displays a window including the contentdisplayed on the main screen, displays the enlarged specific area on themain screen, and displays an indicator indicating the enlarged specificarea within the displayed window. In this case, the content may be avideo.

It will be described in detail with reference to FIG. 37.

When a pointer is located within a predetermined distance from theobject, the controller 3520 displays a guide line indicating a boundaryof the object. Thereafter, after receiving an input for selecting theobject from the remote controller, the controller 3520 specifies theobject within the guide line.

It will be described in detail with reference to FIGS. 38 and 39.

The controller 3520 adjusts the location of the specified specificobject such that the specified specific object is located at the centerof the main screen and enlarges the specific area including the adjustedspecific object.

It will be described in detail with reference to FIG. 40.

When the specific object disappears on the main screen, the controller3520 zooms out on the content currently displayed on the main screen anddisplays the content on the full screen.

It will be described in detail with reference to FIG. 41.

The controller 3520 determines similarity between the displayed newobject and the object stored in the memory by comparing the objects. Ifthe determined similarity is equal to or greater than a predeterminedthreshold, the controller 3520 zooms in on the content currentlydisplayed on the main screen with respect to the object and displays thecontent.

It will be described in detail with reference to FIG. 42.

The controller 3520 receives, from the remote controller, an input forselecting at least one object different from the object displayed on themain screen using the pointer through the interface module 3510.Thereafter, in response to the received input, the controller 3520releases specifying the specific object and then specifies the differentobject(s).

It will be described in detail with reference to FIG. 43.

The controller 3520 adjusts a magnification ratio based on the size ofthe specified object, enlarges a specific area including the objectbased on the adjusted magnification ratio, and displays the enlargedspecific area on the main screen.

It will be described in detail with reference to FIG. 45.

According to an embodiment of the present invention, a specific objectcan be detected from a broadcast signal received from a broadcastingstation, and an enlarged screen for tracing the specific object can bedisplayed.

The interface module 3510 receives the broadcast signal from an externalserver such as the broadcasting station.

The controller 3520 detects the specific object from the receivedbroadcast signal and stores a specific area including the specificobject in a buffer (not shown in the drawing). The controller 3520enlarges the specific area stored in the buffer and displays theenlarged specific area. In this case, the specific area is changedaccording to a location of the specific object.

According to an embodiment of the present invention, only an areaenlarged through a magnifying glass screen can be traced.

When receiving a specific signal from the remote controller, thecontroller 3520 displays a guide line having a shape of a magnifyingglass (not shown in the drawing) on the main screen. In this case, theguide line may have a shape of a tetragon, triangle, circle, or oval,and it can be changed according to a user configuration.

If a specific object within the guide line is specified by the pointer,the controller 3520 specified the specific object within the guide lineand stores features of the specific object in the memory.

When receiving a specific signal for activating a magnifying glassfunction from the remote controller, the controller 3520 adjusts aspecific area including the specific object within the guide line at aspecific ratio and displays the adjusted specific area on the mainscreen. In this case, the specific area is displayed within the guideline. In addition, the specific area may be changed according to alocation of the specific object.

The controller 3520 can enlarge or reduce the specific area.

The controller 3520 can adjust an adjustment ratio of the specific areabased on the features of the object stored in the memory.

FIG. 36 is a flowchart of a control method for a display deviceaccording to an embodiment of the present invention. The method isperformed by the controller 3520.

As shown in FIG. 36, the controller 3520 displays a content including atleast one object on a main screen [S3610].

The controller 3520 enters an enlargement mode in accordance with aninput for requesting the enlargement mode received from a remotecontroller [S3620].

The controller 3520 receives, from the remote controller, an input forselecting a specific object from among the at least one object displayedon the main screen [S3630].

The controller 3520 specifies the specific object in response to thereceived selection input [S3640].

Particularly, when a pointer is located within a predetermined distancefrom the object, the controller 3520 displays a guide line indicating aboundary of the object. Thereafter, after receiving the input forselecting the object from the remote controller, the controller 3520specifies the object within the guide line.

The controller 3520 enlarges a specific area including the specificobject [S3650]. In this case, the specific area may be changed accordingto a location of the specific object.

Particularly, the controller 3520 adjusts the location of the specifiedspecific object such that the specified specific object is located atthe center of the main screen and enlarges the specific area includingthe adjusted specific object.

The controller 3520 displays a window including the content displayed onthe main screen and displays the enlarged specific area on the mainscreen [S3660].

Subsequently, the controller 3520 displays an indicator indicating theenlarged specific area within the displayed window [S3670]. In thiscase, the content may be a video.

FIG. 37 is a diagram illustrating an example of specifying a specificobject and tracing the specified object according to an embodiment ofthe present invention.

Referring to a drawing 3710, the controller 3520 enters an enlargementmode 3712 in accordance with an input for requesting the enlargementmode received from the remote controller, specifies an object such as aball 3714 using a pointer in response to a received selection input, andenlarges a specific area including the specific object such as the ball.In this case, the specific area may be changed according to allocationof the specific object. Thus, when the ball 3714 moves, the specificarea is changed according to the ball movement.

That is, when the controller 3520 receives an input for selecting theball from a user while displaying a soccer game, the controller 3520specifies the ball object, activates the enlargement mode with respectto the location of the ball, and detects the ball object.

As shown in a drawing 3720, the main screen 3720 includes a window 3722,and the window 3722 includes an indicator 3724. As a PIP screen, thewindow 3722 shows the original content before enlargement, and theindicator 3724 shows which portion the enlarged content corresponds toof the original content.

The controller 3520 displays the window 3722 including the contentdisplayed on the main screen, displays the enlarged specific area 3720on the main screen, and displays the indicator 3724 indicating theenlarged specific area. In this case, the content corresponds to avideo. In addition, the indicator 3724 may have a shape of a box,circle, oval, triangle, or diamond. Moreover, the indicator 3724 mayalso have an arbitrary shape according to a user configuration

When the object moves, the controller 3520 detects a movement of theobject. Thereafter, the controller 3520 changes a reference point fromthe original point to the moving point, enlarges a specific areaincluding the object by considering the moving point as the referencepoint and displays the enlarged specific area.

FIG. 38 is a diagram illustrating an example of representing a boundaryof an object adjacent to a pointer as a dotted line when the pointermoves toward the object according to an embodiment of the presentinvention.

Referring to a drawing 3810, when a pointer 3812 is within apredetermined threshold distance from an object such as a ball, thecontroller 3520 displays a guide line 3814 that indicates a boundary ofthe object. Thereafter, the controller 3520 specifies the ball objectwithin the guide line 3814 in accordance with an input for selecting theobject received from the remote controller.

That is, if the pointer is located within the predetermined thresholddistance (e.g., 5 mm) from any one of a plurality of objects displayedon a main screen, the controller 3520 displays the guide line andspecifies the object. In this case, the guide line can be expressed as aspecific color of a dotted line or a flickering line.

As shown in a drawing 3820, the controller 3520 displays a guide line3822 that indicates the boundary of the specific object such as theball, enlarges a specific area including the specific object, anddisplays the enlarged specific area on the screen. The controller 3520displays a window and displays an indicator indicating which portion theenlarged screen corresponds to of the full screen within the window.

FIG. 39 is a diagram illustrating an example of representing a boundaryof an object adjacent to a pointer as a dotted line when the pointermoves toward the object according to an embodiment of the presentinvention.

As shown in a drawing 3910, if a pointer 3914 is located within apredetermined threshold distance from an object such as person, thecontroller 3520 displays a guide line 3912 that indicates a boundary ofthe object. After receiving an input for selecting the object from theremote controller, the controller 3520 specifies the personcorresponding to the object within the guide line 3912.

As shown in a drawing 3920, the controller 3520 displays a guide line3922 that indicates the boundary of the specific object such as theperson, enlarges a specific area including the specific object, anddisplays the enlarged specific area.

According to the present invention, it is possible to increase a rangefor a user to select an object. In addition, When the user selects apreferred object, the user can intuitively know which object isselected, thereby improving user convenience.

FIG. 40 is a diagram illustrating a case in which the center of anobject is located at the center of a main screen according to anembodiment of the present invention.

As shown in drawings 4010 and 4020, a specific object 4012 such as aball moves toward the left side of a main screen.

In this case, the controller 3520 adjusts a location of the specificobject such as the ball to be located at the center 4032 of the mainscreen, enlarges a specific area 4030 including the adjusted specificobject, and displays the enlarged specific area on the main screen.

According to the present invention, when the location of the specificobject is changed, the location of the object can be automaticallyadjusted to be located at the center of the main screen even though auser does not manipulate the remote controller, thereby improving userconvenience.

FIG. 41 is a diagram illustrating a case in which an object moves out ofa screen according to an embodiment of the present invention.

When a specific object 4112 such as a ball moves as shown in a drawing4110, the specific object may disappear on a screen as shown in adrawing 4120.

When the specific object disappears on the main screen, the controller3520 zooms out on a content currently displayed on the main screen andthen displays the content on the full screen. In this case, thecontroller 3520 performs the zoom-out operation during a prescribedtime.

According to the present invention, when the object disappears on thescreen, the enlarged screen is smoothly switched to the full screenduring the prescribed time. Thus, a user is provided with an effect thatthe screen is naturally switched.

FIG. 42 is a diagram illustrating a case in which an object that movesout of a screen appears again according to an embodiment of the presentinvention.

As shown in a drawing 4210, after an object moves out of a screen, theobject appears on the screen again.

The controller 3520 stores features of a specific object in the memory3540.

The features of the object include at least one of a shape, a color, anda motion vector. For example, when the current channel is a sportschannel, the object features include at least one of a player's uniformnumber, a uniform shape, a uniform color, and a face shape.

The controller 3520 determines similarity between the displayed newobject and the object stored in the memory by comparing the objects.

Particularly, the controller 3520 determines the similarity withreference to at least one of the shapes, colors, and motion vectors ofthe objects. The controller 3520 determines the similarity bydetermining whether the objects have the same shape or color.

The motion vector means a movement direction of the object. That is, itis assumed that there is a wall in the middle of the screen. If thespecific object moves toward the right and passes through the rear sideof the wall, the specific object disappears on the screen. If thespecific object continuously moves toward the right, the specific objectappears again on the screen.

Thus, the controller 3520 can determine the similarity in considerationsuch as directivity.

If the determined similarity is equal to or greater than a predeterminedthreshold, the controller 3520 zooms in on the content currentlydisplayed on the main screen with respect to a specific object 4222 anddisplays the content as shown in a drawing 4220.

For example, when a ball appears again on the screen, the controller3520 determines similarity with reference to at least one of shapes,colors, and motion vectors of balls. If the determined similarity isequal to or greater than 80%, the controller 3520 determines that a balldisplayed on the previous screen and the ball displayed on the currentscreen are the same. Thereafter, the controller 3520 zooms in on thecurrent content displayed on the main screen with respect to the balland then displays the content as shown in the drawing 4220.

When the specific object appears again on the main screen, thecontroller 3520 zooms in on the content currently displayed on the mainscreen, i.e., from the full screen to the enlarged screen and thendisplays the content as shown in the drawing 4220. In this case, thecontroller 3520 performs the zoom-in operation during a prescribed time.

According to the present invention, when the object appears again on thescreen, the full screen is smoothly switched to the enlarged screenduring the prescribed time. Thus, a user is provided with the effectthat the screen is naturally switched.

FIG. 43 is a diagram illustrating an example of specifying a differentobject and tracing the specified different object while displaying acontent according to an embodiment of the present invention.

When the controller 3520 receives, from the remote controller, an inputfor selecting at least one object different from a specific objectdisplayed on the main screen, the controller 3520 releases specifyingthe specific object and then specifies the different object(s) inresponse to the received input.

For example, the controller 3520 receives, from the remote controller,an input for selecting an object 4332 such as a person different from aspecific object 4312 such as a ball displayed on the main screen shownin a drawing 4310 using a pointer 4322.

In response to the received input, the controller 3520 releasesspecifying the ball object 4312 and then specifies the person object4332.

As shown in a drawing 4340, the controller 3520 adjusts the screen suchthat the person object is located at the center 4342 of the main screenand then displays the adjusted screen.

FIG. 44 is a diagram illustrating a case in which an object has a smallsize or an object cannot be recognized according to an embodiment of thepresent invention.

As shown in a drawing 4410, the controller 3520 checks a recognitionrate that represents a level capable of recognizing an object 4412. Ifthe checked recognition rate is lower than a threshold value, thecontroller 3520 initializes specifying the object.

For example, if the checked recognition rate is lower than 70%, thecontroller 3520 initializes specifying the object.

In addition, if a size of the object is smaller than a predeterminedthreshold, the controller 3520 initializes specifying the object.

For example, if the size of the object is smaller than 1 cm, thecontroller 3520 initializes specifying the object.

The controller 3520 determines a range of specifiable object sizes. Whenan object appearing on the screen does not move out of a zoom-in screen,the controller 3520 detects the object irrespective of a size of theobject, specifies the detected object, and traces the specified object.

If the object moves out of the zoom-in screen, the controller 3520performs the zoom-out operation during the prescribed time to displaythe full screen and initializes specifying the object.

As shown in the drawing 4410, the controller 3520 specifies the object4412. When the object appearing on the screen does not move out of thezoom screen as shown in a drawing 4420, the controller 3520 continuouslydetects the object and then specifies the detected object.

When the object moves out of the screen as shown in a drawing 4430(4432), the controller 3520 performs the zoom-out operation during theprescribed time to display the full screen.

As shown in a drawing 4440, if the recognition rate of the object islower than the predetermined threshold after the zoom-out operation isperformed, the controller 3520 initializes specifying the object andreceives an input for specifying a new object.

FIG. 45 is a diagram illustrating an example of adjusting a zoom ratioand a position for each object according to an embodiment of the presentinvention.

The controller 3520 adjusts a magnification ratio based on a size of aspecified specific object, enlarges a specific area including the objectbased on the adjusted magnification ratio, and displays the enlargedspecific area on the main screen.

As shown in a drawing 4510, the controller 3520 adjusts the screenmagnification ratio according to a size 4512 of the detected object,enlarges a specific area 4512 including the object based on themagnification ratio scaled by, for example, two times, and then displaysthe specific area on the main screen.

In addition, as shown in a drawing 4520, when the size of the object4522 is changed due to a change in the screen, the controller 3520adjusts the screen magnification ratio according to the changed size ofthe object, enlarges a specific area 4522 including the object based onthe magnification ratio scaled by, for example, three times, and thendisplays the specific area on the main screen.

When increasing or decreasing the screen magnification ratio, thecontroller 3520 adjusts times required for the zoom-in and zoom-outoperations.

According to the present invention, when the zoom-in and zoom-outoperations are performed, a user can be provided with the effect thatthe screen is naturally switched, thereby improving user convenience.

In addition, even when the specific object moves out of the main screen,the controller 3520 adjusts the screen magnification ratio inconsideration of the object size.

According to an embodiment of the present invention, the controller 3520can recognize an object and adjust a size and shape of an enlargedscreen based on features of the recognized object.

The object may include a person, a building, a vehicle, etc. Thecontroller 3520 can adjust the size and shape of the enlarged screen tobe optimized for the object based on the features of the object.

For example, when the object is a person, the controller 3520 adjuststhe enlarged screen based on features of the person object such that abody and face of the person are displayed well.

When the object is a building, the controller 3520 adjusts the enlargedscreen based on features of the building object such that a buildingentrance and a shape of the entire building are displayed well.

The shape of the enlarged screen includes not only a tetragon but alsovarious shapes such as an oval, circle, triangle, and polygon.Additionally, it can be randomly configured by the user.

FIG. 46 is a diagram illustrating an example of specifying multipleobjects and tracing the specified multiple objects according to anembodiment of the present invention.

When the controller 3520 receives an input for selecting objectsdifferent from a specified specific object using a pointer from theremote controller, the controller 3520 displays an additional selectionmessage. In addition, when receiving an additional selection input fromthe remote controller, the controller 3520 performs grouping of theobjects different from the specified object and then displays thegrouped objects.

For example, as shown in a drawing 4610, when the controller 3520receives an input for selecting an object 4612 such as a specific playerand an object 4618 such as a ball using a pointer 4614 from the remotecontroller, the controller 3520 displays an additional selection messagebox 4616. In this case, the additional selection message box may be apop-up window.

If the pointer is placed over an object that will be additionallyselected after the controller 3520 specifies the specific object, thecontroller 3520 displays the additional selection message box throughthe pop-up window.

As shown in a drawing 4620, when receiving an additional selection inputfrom the remote controller, the controller 3520 performs grouping 4622of the specific player object and the ball object and then displays thegrouped objects on the main screen.

The controller 3520 groups the ball object and the player object as asingle group, enlarges the grouped objects, and displays the enlargedobjects on the main screen.

If either the ball object or the player object disappears on the screen,the controller 3520 releases the group. Thereafter, the controller 3520enlarges a specific area with respect to the object that remains on thescreen and then displays the enlarged specific area on the main screen.

If both of the ball object and the player object disappear, thecontroller 3520 releases the group. Thereafter, the controller 3520initializes a mode for specifying an object.

FIG. 47 is a diagram illustrating an example of specifying multipleobjects and tracing the specified multiple objects according to anembodiment of the present invention.

As shown in a drawing 4710, the controller 3520 specifies a firstobject, a second object, and a third object and then enlarges a specificarea including the specified first, second and third objects.

As shown in a drawing 4720, the controller displays the enlargedspecific area on the main screen. If the first to third objects move,the controller 3520 traces the specific area including the first tothird objects and displays the traced specific area.

FIG. 48 is a diagram illustrating an example of specifying any one amongmultiple objects and performing tracing with respect to the specifiedobject according to an embodiment of the present invention.

If a pointer 4812 is placed over an object as shown in a drawing 4810,the controller 3520 generates indicators over objects as shown in adrawing 4820

If the controller 3520 receives an input for selecting a fourth objectusing the pointer 4812 from the remote controller as shown in thedrawing 4820, the controller adjusts the screen such that the fourthobject is located at the center of the screen, enlarges a specific areaincluding not only the fourth object but also other objects with respectto the fourth object, and then displays the enlarged specific area onthe main screen.

When the fourth object moves to a different location, the controller3520 adjusts the screen again such that the fourth object is located atthe center of the screen.

In this case, the controller 3520 may maintain a screen ratio to be thesame as that before specifying the object. Alternatively, the controller3520 may adjust the screen ratio based on a size of the specifiedobject.

FIG. 49 is a diagram illustrating an example in which when an object isa person, the object is divided according to parts of the body and adifferent magnification ratio is applied depending on the selected partaccording to an embodiment of the present invention.

When an object is a person as shown in a drawing 4910, the controller3520 can divide the object according to parts of the body and thenenlarge an area including the object to be optimized for a size of theobject.

For example, the controller 3520 divides the person object into the faceand whole body. In addition, if an area corresponding to the personobject is specified using a pointer 4912, the controller may divide theobject into specific parts. Particularly, the controller 3520 may dividethe person object into at least one of the face, whole body, leg, arm,torso, chest, abdomen, ankle, wrist, etc.

As shown in a drawing 4920, when the controller 3520 receives an inputfor specifying the face using the pointer 4912 from the remotecontroller, the controller 3520 enlarges a specific area including theface and then displays the enlarged specific area on the main screen.

As shown in a drawing 4930, when the controller 3520 receives an inputfor specifying the whole body using the pointer 4912 from the remotecontroller, the controller 3520 enlarges a specific area including thewhole body and then displays the enlarged specific area on the mainscreen.

FIG. 50 is a diagram illustrating an example of specifying an object ina different manner depending on movement of the object according to anembodiment of the present invention.

As shown in a drawing 5000, in the case of a baseball game, players movea lot, but spectators rarely move.

The controller 3520 detects a movement of an object. If the detectedchange in the object movement is lower than a predetermined threshold,the controller 3520 recognizes the object as a spectator as shown in adrawing 5010. On the contrary, if the detected change in the objectmovement is equal to or higher than the predetermined threshold, thecontroller 3520 recognizes the object as a baseball player.

That is, the controller 3520 can recognize the object as either thespectator or the baseball player based on the detected change in theobject movement and specify the moving object based on features of theobject.

As shown in a drawing 5030, if an area located within a predetermineddistance from a baseball object is specified, the controller 3520recognizes that the baseball is specified. Thereafter, the controller3520 enlarges a specific area including the specified ball and thendisplays the enlarged specific area.

Specifically, in the case of an object that moves at a high speed suchas a ball, it is difficult to specify the object using the point.According to the present invention, if an area including the ball objectis specified, the controller 3520 may recognize that the ball object isspecified. In other words, a user does not need to specify the ball withthe high speed, thereby improving user convenience.

In addition, according to the present invention, even if an object isnot specified using the pointer, the object can be automaticallyspecified based on a detected change in the object movement, therebyimproving user convenience.

FIG. 51 is a diagram illustrating an example in which when multipleobjects are specified, the specified multiple objects are displayed onPBP screens according to an embodiment of the present invention.

As shown in a drawing 5110, if the controller 3520 receives inputs forspecifying a plurality of objects, i.e., first, second, third objectsfrom the remote controller, the controller 3520 enlarges the first,second, and third objects as shown in drawings 5120, 5130, and 5140,respectively and displays specific areas including the enlarged objectson picture-by-picture (PBP) screens, which are located at the rightside.

In this case, locations of the PBP screen can be adjusted, and each ofthe enlarged specific areas can displayed on the main screen as a PIPscreen.

FIG. 52 is a diagram illustrating an example in which a user searchesfor a specific object on a content through association with time shiftand search functions according to an embodiment of the presentinvention.

As shown in a drawing 5200, in the case of a movie content where aspecific actor appears, the controller 3520 can receive an input forspecifying an object as the specific actor 5220 using a pointer 5210. Inthis case, the controller 3520 retrieves contents stored in the memorythrough association with a time shift function, detects a specificsection including a specific object such as the specific actor based onfeatures of the specific actor object, enlarges the detected specificsection, and then displays the enlarged specific section.

According to the conventional VOD service, when a function of searchingfor a specific actor is performed, only names of a movie, anadvertisement, and a drama show where the specific actor appears areprovided. However, according to the present invention, it is possible toprovide a function of retrieving a specific section of the content suchas a movie where the specific actor appears, enlarging the retrievedsection, and displaying the enlarged section.

In other words, unlike the related art where only the names areprovided, the present invention can provide particular informationnecessary for a user, that is, by displaying only the section where thespecific actor appears, thereby improving user convenience.

According to an embodiment of the present invention, it is possible tospecify a specific object such as a ball on the screen, enlarge aspecific area including the specified object, display the enlargedspecific area by continuously tracing the specified object, therebyimproving user convenience.

According to another embodiment of the present invention, when a pointerapproaches an object, a boundary of the object is displayed as a guideline, and thus, a user can easily specify the object. Therefore, userconvenience is improved.

According to a further embodiment of the present invention, when a userinput no command through the remote controller after checking a locationof a specific object, the specific object can be displayed on the centerof the screen, and thus, the user can watch the object on the center ofthe screen, thereby improving user convenience.

According to a still further object of the present invention, when acurrently displayed object moves out of the screen, a zoom-out operationis performed, that is, a full screen is displayed. Thereafter, when theobject appears again on the screen, a zoom-in operation is performed,thereby proving a user with the effect that the screen is naturallyswitched. Therefore, user convenience can be improved.

The image display device and operating method therefor disclosed in thepresent specification are not limited to the configurations and methodsdescribed above with reference to the embodiments. Moreover, some or allof the embodiments are selectively combined for various modifications.

Meanwhile, the operating method for the image display device accordingto the present invention may be implemented as code that can be writtenon a processor-readable recording medium and thus read by a processorprovided in a network device. The processor-readable recording mediummay include all kinds of recording media where data that can be read bythe processor are stored. The processor-readable recording medium mayinclude, for example, a ROM, a RAM, a CD-ROM, a magnetic tape, a floppydisk, and an optical data memory and it may be implemented in the formof a carrier wave transmitted over the Internet. In addition, theprocessor-readable recording medium may be distributed over a pluralityof computer systems connected to a network such that processor-readablecode is written thereto and executed therefrom in a decentralizedmanner.

Although the present invention has been described with reference to thepreferred embodiments thereof, the invention is not limited to theaforementioned specific embodiments. In addition, it will be understoodby those skilled in the art that various changes in form and detail maybe made without departing from the spirit and scope of the invention. Itis also apparent that such variations of this specification are not tobe understood individually or separately from the technical scope orspirit of this specification.

Mode for Carrying Out Invention

Hereinabove, various embodiments have been described for the best modefor implementing the present invention.

INDUSTRIAL APPLICABILITY

The present invention can be applied to various display devices.

It is apparent to those skilled in the art that various modificationsand changes may be made without departing from the spirit and scope ofthe present invention. Accordingly, the present invention is intended toinclude the modifications and changes of the present invention withinthe attached claims and the equivalent scope.

What is claimed is:
 1. A television controlled by a remote controller,the television comprising: an user input interface configured to receiveat least one control signal from the remote controller; a tunerconfigured to receive a broadcast signal of a channel selected by the atleast one control signal from the remote controller; a decoderconfigured to decode video data included in the broadcast signal; adisplay configured to display the video data in a full screen; acontroller configured to detect a specific object included in the videodata, wherein the specific object includes a person, enlarge a specificarea including the specific object to be optimized for a size of thespecific object, control the display to display, on an upper side of thefull screen, a notification message for indicating that a specific areaenlargement mode is activated, control the display to display, on aright side of the full screen, the enlarged specific area including thespecific object, and control the display to display, on a left side ofthe full screen, the video data with an indicator for selecting andenlarging at least one part of the person's body.
 2. The television ofclaim 1, wherein the remote controller stores information on a frequencyband for pairing with the television.
 3. The television of claim 1,wherein the at least one part of the person's body includes a face, aleg, an arm, a torso, a chest, an abdomen, an ankle or a wrist.
 4. Thetelevision of claim 1, wherein the specific area enlargement mode isautomatically activated when the video data relates to a specificcategory.
 5. The television of claim 4, wherein the specific categoryincludes news or sports.
 6. A method of processing data in a televisioncontrolled by a remote controller, the method comprising: receiving atleast one control signal from the remote controller; receiving abroadcast signal of a channel selected by the at least one controlsignal from the remote controller; decoding video data included in thebroadcast signal; displaying the video data in a full screen; detectinga specific object included in the video data, wherein the specificobject includes a person; enlarging a specific area including thespecific object to be optimized for a size of the specific object;displaying, on an upper side of the full screen, a notification messagefor indicating that a specific area enlargement mode is activated;displaying, on a right side of the full screen, the enlarged specificarea including the specific object; and displaying, on a left side ofthe full screen, the video data with an indicator for selecting andenlarging at least one part of the person's body.
 7. The method of claim6, wherein information on a frequency band for pairing with thetelevision is stored in the remote controller.
 8. The method of claim 6,wherein the at least one part of the person's body includes a face, aleg, an arm, a torso, a chest, an abdomen, an ankle or a wrist.
 9. Themethod of claim 1, further comprising: activating the specific areaenlargement mode automatically when the video data relates to a specificcategory.
 10. The method of claim 9, wherein the specific categoryincludes news or sports.